Warning: Insecure world writable dir while running Ruby in SublimeREPL - ruby

I am trying to run ruby from sublimeREPL. My configuration files are as follows:
Packages --> SublimeREPL --> config --> Ruby --> pry_repl.rb
require 'rubygems'
gem 'pry'
require 'pry'
require 'pry/input_completer'
require 'socket'
require 'thread'
require 'json'
include Socket::Constants
class PryInput
def readline(prompt)
$stdout.print prompt
$stdout.flush
$stdin.readline
end
end
class PryOutput
# def puts(data="")
def print(data="")
$stdout.puts(data.gsub('`', "'"))
$stdout.flush
end
end
Pry.config.input = PryInput.new()
Pry.config.output = PryOutput.new()
Pry.config.color = false
Pry.config.editor = ARGV[0]
Pry.config.auto_indent = false
Pry.config.correct_indent = false
port = ENV["SUBLIMEREPL_AC_PORT"].to_i
socket = Socket.new(AF_INET, SOCK_STREAM, 0)
sockaddr = Socket.pack_sockaddr_in(port, '127.0.0.1')
socket.connect(sockaddr)
# completer = Pry::InputCompleter.build_completion_proc(binding)
completer = Pry::InputCompleter.new(binding)
def read_netstring(s)
size = 0
while true
ch = s.recvfrom(1)[0]
if ch == ':'
break
end
size = size * 10 + ch.to_i
end
msg = ""
while size != 0
msg += s.recvfrom(size)[0]
size -= msg.length
end
ch = s.recvfrom(1)[0]
return msg
end
# Thread.abort_on_exception = true
t1 = Thread.new do
while true
data = read_netstring(socket)
req = JSON.parse(data)
line = req["line"]
completions = completer.call(req["line"])
response = [line, completions]
response_msg = JSON.dump(response)
payload = response_msg.length.to_s + ":" + response_msg + ","
socket.write(payload)
end
end
Pry.start self
Packages --> SublimeREPL --> config --> Ruby --> Main.sublime-build
[
{
"id":"tools",
"children":[
{
"caption":"SublimeREPL",
"mnemonic":"R",
"id":"SublimeREPL",
"children":[
{
"caption":"Ruby",
"children":[
{
"command":"repl_open",
"caption":"Ruby",
"id":"repl_ruby",
"mnemonic":"R",
"args":{
"type":"subprocess",
"external_id":"ruby",
"encoding":"utf8",
"cmd":{
"windows":[
"ruby.exe",
"${packages}/SublimeREPL/config/Ruby/pry_repl.rb",
"$editor"
],
"linux":[
"/usr/bin/ruby",
"${packages}/SublimeREPL/config/Ruby/pry_repl.rb",
"$editor"
],
"osx":[
"ruby",
"${packages}/SublimeREPL/config/Ruby/pry_repl.rb",
"$editor"
]
},
"soft_quit":"\nexit\n",
"cwd":"$file_path",
"cmd_postfix":"\n",
"autocomplete_server": true,
"syntax":"Packages/Ruby/Ruby.tmLanguage"
}
},
{
"command":"repl_open",
"caption":"Ruby - IRB (deprecated)",
"id":"repl_ruby_irb",
"mnemonic":"I",
"args":{
"type":"subprocess",
"external_id":"ruby",
"encoding":"utf8",
"cmd":{
"windows":[
"irb.bat",
"--noreadline",
"--inf-ruby-mode"
],
"linux":[
"irb",
"--noreadline",
"--inf-ruby-mode"
],
"osx":[
"irb",
"--noreadline",
"--inf-ruby-mode"
]
},
"soft_quit":"\nexit\n",
"cwd":"$file_path",
"cmd_postfix":"\n",
"suppress_echo":true,
"syntax":"Packages/Ruby/Ruby.tmLanguage"
}
}
]
}
]
}
]
}
]
Tools --> Build System --> New build system --> sublimereplruby.sublime-build
{
"target": "run_existing_window_command",
"id": "repl_ruby",
"file": "config/Ruby/Main.sublime-menu"
}
When I try to run any .rb file by selecting the above sublimereplruby.sublime-build build system, I get this output:
[1] pry(main)>
and upon hitting Enter, I get this:
/var/lib/gems/2.5.0/gems/pry-0.12.2/lib/pry/pager.rb:150: warning: Insecure world writable dir /usr/lib/jvm/java-8-openjdk-amd64/bin in PATH, mode 040777
=> nil
[2] pry(main)>
I do not get any output. I get only nil. I started learning Ruby today. So I do not know about gem and related topics. I need to run Ruby in Sublime Text3. So please tell me how I can rectify the problem.

Related

How to merge multiple hashes?

Right now, I'm merging two hashes like this:
department_hash = self.parse_department html
super_saver_hash = self.parse_super_saver html
final_hash = department_hash.merge(super_saver_hash)
Output:
{:department=>{"Pet Supplies"=>{"Birds"=>16281, "Cats"=>245512,
"Dogs"=>513926, "Fish & Aquatic Pets"=>46811, "Horses"=>14805,
"Insects"=>364, "Reptiles & Amphibians"=>5816, "Small
Animals"=>19769}}, :super_saver=>{"Free Super Saver
Shipping"=>126649}}
But now I want to merge more in the future. For example:
department_hash = self.parse_department html
super_saver_hash = self.parse_super_saver html
categories_hash = self.parse_categories html
How to merge multiple hashes?
You can do below way using Enumerable#inject:
h = {}
arr = [{:a=>"b"},{"c" => 2},{:a=>4,"c"=>"Hi"}]
arr.inject(h,:update)
# => {:a=>4, "c"=>"Hi"}
arr.inject(:update)
# => {:a=>4, "c"=>"Hi"}
How about:
[department_hash, super_saver_hash, categories_hash].reduce &:merge
You can just call merge again:
h1 = {foo: :bar}
h2 = {baz: :qux}
h3 = {quux: :garply}
h1.merge(h2).merge(h3)
#=> {:foo=>:bar, :baz=>:qux, :quux=>:garply}
It took me a while to figure out how to merge multi-nested hashes after going through this Question and its Answers. It turned out I was iterating through the collections of hashes incorrectly, causing all kinds of problems with null values.
This sample command-line app shows how to merge multiple hashes with a combination of store and merge!, depending on whether or not they were top-level hash keys. It uses command-line args with a few known key name for categorization purposes.
Full code from the Gist URL is provided below as a courtesy:
# Ruby - A nested hash example
# Load each pair of args on the command-line as a key-value pair
# For example from CMD.exe:
# call ruby.exe ruby_nested_hash_example.rb Age 30 Name Mary Fav_Hobby Ataraxia Fav_Number 42
# Output would be:
# {
# "data_info": {
# "types": {
# "nums": {
# "Age": 30,
# "Fav_Number": 42
# },
# "strings": {
# "Name": "Mary",
# "Fav_Hobby": "Ataraxia"
# }
# },
# "data_id": "13435436457"
# }
# }
if (ARGV.count % 2 != 0) || (ARGV.count < 2)
STDERR.puts "You must provide an even amount of command-line args to make key-value pairs.\n"
abort
end
require 'json'
cmd_hashes = {}
nums = {}
strings = {}
types = {}
#FYI `tl` == top-level
all_tl_keys = {}
data_info = {}
data_id = {:data_id => "13435436457"}
_key = ""
_value = ""
element = 0
ARGV.each do |i|
if element % 2 == 0
_key=i
else
if (i.to_i!=0) && (i!=0)
_value=i.to_i
else
_value=i
end
end
if (_key != "") && (_value != "")
cmd_hashes.store(_key, _value)
_key = ""
_value = ""
end
element+=1
end
cmd_hashes.each do |key, value|
if value.is_a? Numeric
nums.store(key, value)
else
strings.store(key, value)
end
end
if nums.size > 0; types.merge!(:nums => nums) end
if strings.size > 0; types.merge!(:strings => strings) end
if types.size > 0; all_tl_keys.merge!(:types => types) end
if data_id.size > 0; all_tl_keys.merge!(data_id) end
if all_tl_keys.size > 0; data_info.merge!(:data_info => all_tl_keys) end
if data_info.size > 0; puts JSON.pretty_generate(data_info) end
Suppose you are having arr = [{x: 10},{y: 20},{z: 30}]
then do
arr.reduce(:merge)

Ruby Git Diff Line Information Parser

How can I parse the output of a git diff and get line information (i.e. which lines has been added/modified)?
I would like something similar to
raw = `git diff`
parsed = Git.Diff.parse(raw)
parsed.each do |file|
file.each do |line|
puts "#{file.name} - #{line.number} - #{line.type}"
end
end
Edit:
Sample output
[
{
"file": "path/to/file1",
"lines": [
{ number: "1", type: "modified"},
{ number: "4", type: "deleted"},
{ number: "9", type: "added"}
]
},
{
"file": "path/to/file2",
"lines": [
{ number: "4", type: "modified"},
{ number: "5", type: "added"}
]
}
]
What you need is to correctly group the output in file chunks and keep what is needed.
Getting the diff
You can get it by simply running a
`git --diff`
What lines are needed?
lines starting with 'diff --git' from where you can get the file's name
lines starting with '+ ' that are the added ones
lines starting with '- ' that are the removed ones
How to group them?
For these things Enumerable#slice_before comes to mind.
Putting it together
I ended up with this prototype:
raw_data = `git diff`.split("\n")
# Keep what is needed
clean_data = raw_data.select { |li|
li.starts_with?('diff --git') ||
li.starts_with?('- ') ||
li.starts_with?('+ ')
}
# Group the by file
# [[file_1, line1, line2, line3], [file_2, line1]]
file_data = clean_data.slice_before { |li| li.starts_with?('diff --git') }
# This is the output format
output = Hash.new {|h,k| h[k] = { added: 0, removed: 0 } }
# Populate the output
file_data.each_with_object(output) do |f_data, memo|
file, *file_info = f_data
file = file.split(' b/').first.gsub('diff --git a/', '')
file_info.each { |f_info|
memo[file][f_info[0] == '+' ? :added : :removed] += 1
}
end
Output example
{
"file_1" => { added: 1, removed: 12 },
"file_2" => { added: 0, removed: 1 }
}
I am sure it can get better :-)
Here is what I ended up with
class Parser
def parse(text)
if text.encoding.name != "UTF-8"
encoded_text = #full_diff.encode("UTF-8", "binary", { :invalid => :replace, :undef => :replace })
else
encoded_text = text
end
hunks = []
hunk = nil
added_line_number = nil
deleted_line_number = nil
lines = encoded_text.strip.split("\n")
lines.each_with_index do |line, index|
if m = /^diff --git a\/(.*?) b\/(.*?)$/.match(line)
raise "Diff formatting error, 'diff --git' is the last line" if index + 1 >= lines.length
# new hunk
added_line_number = nil
delete_line_number = nil
hunk = Hunk.new(m[1], m[2])
hunk.type = hunk_type(lines[index + 1], m[1], m[2])
hunks.push(hunk)
elsif /^Binary files /.match(line)
hunk.is_binary = true
elsif m = /^## \-(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? ##/.match(line)
# (e.g. ## -19,6 +19,7 ##)
deleted_line_number = Integer(m[1])
added_line_number = Integer(m[2])
else
if !added_line_number.nil?
if line.start_with?('+')
# added line
hunk.lines.push SourceLine.new(added_line_number, SourceLine::Type::Added, line[1..-1])
added_line_number += 1
elsif line.start_with?('-')
# deleted line
hunk.lines.push SourceLine.new(deleted_line_number, SourceLine::Type::Deleted, line[1..-1])
deleted_line_number += 1
else
# unmodified line
added_line_number += 1
deleted_line_number += 1
end
end
end
end
hunks
end
def hunk_type(line, original, renamed)
case line
when /^new file /
type = Hunk::Type::Added
when /^deleted file /
type = Hunk::Type::Deleted
else
type = original == renamed ? Hunk::Type::Modified : Hunk::Type::Renamed
end
type
end
private :hunk_type
end
end
module Type
Added = 'added'
Deleted = 'deleted'
Modified = 'modified'
Renamed = 'renamed'
end
class Hunk
module Type
Added = 'added'
Deleted = 'deleted'
Modified = 'modified'
Renamed = 'renamed'
end
attr_accessor :original_path, :renamed_path, :type, :lines, :is_binary
alias_method :is_binary?, :is_binary
def initialize(original_path, renamed_path)
self.is_binary = false
self.lines = []
self.original_path = original_path
self.renamed_path = renamed_path
end
end
class SourceLine
module Type
Added = 'added'
Deleted = 'deleted'
end
attr_accessor :number, :type, :text
def initialize(number, type, text)
self.number = number
self.type = type
self.text = text
end
end

Why do these threads stop working?

Since Shopify's default products importer (via CSV) is really slow, I'm using multithreading to add ~24000 products to a Shopify store using the API. The API has a call limit of 2 per second. With 4 threads the calls are within the limit.
But after a while all threads stop working except one. I don't get any error messages, the code keeps running but doesn't print any product information. I can't seem to figure out what's going wrong.
Here's the code I'm using:
require 'shopify_api'
require 'open-uri'
require 'json'
require 'base64'
begin_time = Time.now
my_threads = []
shop_url = "https://<API_KEY>:<PASSWORD>#<SHOPNAME>.myshopify.com/admin"
ShopifyAPI::Base.site = shop_url
raw_product_data = JSON.parse(open('<REDACTED>') {|f| f.read }.force_encoding('UTF-8'))
# Split raw product data
one, two, three, four = raw_product_data.each_slice( (raw_product_data.size/4.0).round ).to_a
def category_to_tag(input)
<REDACTED>
end
def bazookah(array, number)
array.each do |item|
single_product_begin_time = Time.now
# Store item data in variables
vendor = item['brand'].nil? ? 'Overige' : item['brand']
title = item['description']
item_size = item['salesUnitSize']
body = "#{vendor} - #{title} - #{item_size}"
type = item['category'].nil? ? 'Overige' : item['category']
tags = category_to_tag(item['category']) unless item['category'].nil?
variant_sku = item['itemId']
variant_price = item['basePrice']['price']
if !item['images'].nil? && !item['images'][2].nil?
image_src = item['images'][2]['url']
end
image_time_begin = Time.now
image = Base64.encode64(open(image_src) { |io| io.read }) unless image_src.nil?
image_time_end = Time.now
total_image_time = image_time_end - image_time_begin
# Create new product
new_product = ShopifyAPI::Product.new
new_product.title = title
new_product.body_html = body
new_product.product_type = type
new_product.vendor = vendor
new_product.tags = item['category'].nil? ? 'Overige' : tags
new_product.variants = [ <REDACTED> ]
new_product.images = [ <REDACTED> ]
new_product.save
creation_time = Time.now - single_product_begin_time
puts "#{number}: #{variant_sku} - P: #{creation_time.round(2)} - I: #{image_src.nil? ? 'No image' : total_image_time.round(3)}"
end
end
puts '====================================================================================='
puts "#{raw_product_data.size} products loaded. Starting import at #{begin_time}..."
puts '-------------------------------------------------------------------------------------'
my_threads << Thread.new { bazookah(one, 'one') }
my_threads << Thread.new { bazookah(two, 'two') }
my_threads << Thread.new { bazookah(three, 'three') }
my_threads << Thread.new { bazookah(four, 'four') }
my_threads.each { |thr| thr.join }
puts '-------------------------------------------------------------------------------------'
puts "Done. It took #{Time.now - begin_time} minutes."
puts '====================================================================================='
What could I try to solve this?
It most likely has something to do with this:
http://docs.shopify.com/api/introduction/api-call-limit
I'd suspect that you are being rate limited by Shopify. You are trying to add 24000 records, via the API, from a single IP address. Most people don't like that kind of thing.

Ruby Watir: cannot launch browser in a thread in Linux

I'm trying to run this code in Red Hat Linux, and it won't launch a browser. The only way I can get it to work is if i ALSO launch a browser OUTSIDE of the thread, which makes no sense to me. Here is what I mean:
require 'watir-webdriver'
$alphabet = ["A", "B", "C"]
$alphabet.each do |z|
puts "pshaw"
Thread.new{
Thread.current["testPuts"] = "ohai " + z.to_s
Thread.current["myBrowser"] = Watir::Browser.new :ff
puts Thread.current["testPuts"] }
$browser = Watir::Browser.new :ff
end
the output is:
pshaw
(launches browser)
ohai A
(launches browser)
pshaw
(launches browser)
ohai B
(launches browser)
pshaw
(launches browser)
ohai C
(launches browser)
However, if I remove the browser launch that is outside of the thread, as so:
require 'watir-webdriver'
$alphabet = ["A", "B", "C"]
$alphabet.each do |z|
puts "pshaw"
Thread.new{
Thread.current["testPuts"] = "ohai " + z.to_s
Thread.current["myBrowser"] = Watir::Browser.new :ff
puts Thread.current["testPuts"] }
end
The output is:
pshaw
pshaw
pshaw
What is going on here? How do I fix this so that I can launch a browser inside a thread?
EDIT TO ADD:
The solution Justin Ko provided worked on the psedocode above, but it's not helping with my actual code:
require 'watir-webdriver'
require_relative 'Credentials'
require_relative 'ReportGenerator'
require_relative 'installPageLayouts'
require_relative 'PackageHandler'
Dir[(Dir.pwd.to_s + "/bmx*")].each {|file| require_relative file } #this includes all the files in the directory with names starting with bmx
module Runner
def self.runTestCases(orgType, *caseNumbers)
$testCaseArray = Array.new
caseNumbers.each do |thisCaseNum|
$testCaseArray << thisCaseNum
end
$allTestCaseResults = Array.new
$alphabet = ["A", "B", "C"]
#count = 0
#multiOrg = 0
#peOrg = 0
#eeOrg = 0
#threads = Array.new
$testCaseArray.each do |thisCase|
$alphabet[#count] = Thread.new {
puts "working one"
Thread.current["tBrowser"] = Watir::Browser.new :ff
puts "working two"
if ((thisCase.declareOrg().downcase == "multicurrency") || (thisCase.declareOrg().downcase == "mc"))
currentOrg = $multicurrencyOrgArray[#multiOrg]
#multiOrg += 1
elsif ((thisCase.declareOrg().downcase == "enterprise") || (thisCase.declareOrg().downcase == "ee"))
currentOrg = $eeOrgArray[#eeOrg]
#eeOrg += 1
else #default to single currency PE
currentOrg = $peOrgArray[#peOrg]
#peOrg += 1
end
setupOrg(currentOrg, thisCase.testCaseID, currentOrg.layoutDirectory)
runningTest = thisCase.actualTest()
if runningTest.crashed != "crashed" #changed this to read the attr_reader isntead of the deleted caseStatus method from TestCase.rb
cleanupOrg(thisCase.testCaseID, currentOrg.layoutDirectory)
end
#threads << Thread.current
}
#count += 1
end
#threads.each do |thisThread|
thisThread.join
end
writeReport($allTestCaseResults)
end
def self.setupOrg(thisOrg, caseID, layoutPath)
begin
thisOrg.logIn
pkg = PackageHandler.new
basicInstalled = "false"
counter = 0
until ((basicInstalled == "true") || (counter == 5))
pkg.basicInstaller()
if Thread.current["tBrowser"].text.include? "You have attempted to access a page"
thisOrg.logIn
else
basicInstalled = "true"
end
counter +=1
end
if !((caseID.include? "bmxb") || (caseID.include? "BMXB"))
moduleInstalled = "false"
counter2 = 0
until ((moduleInstalled == "true") || (counter == 5))
pkg.packageInstaller(caseID)
if Thread.current["tBrowser"].text.include? "You have attempted to access a page"
thisOrg.logIn
else
moduleInstalled = "true"
end
counter2 +=1
end
end
installPageLayouts(layoutPath)
rescue
$allTestCaseResults << TestCaseResult.new(caseID, caseID, 1, "SETUP FAILED!" + "<p>#{$!}</p><p>#{$#}</p>").hashEmUp
writeReport($allTestCaseResults)
end
end
def self.cleanupOrg(caseID, layoutPath)
begin
uninstallPageLayouts(layoutPath)
pkg = PackageHandler.new
pkg.packageUninstaller(caseID)
Thread.current["tBrowser"].close
rescue
$allTestCaseResults << TestCaseResult.new(caseID, caseID, 1, "CLEANUP FAILED!" + "<p>#{$!}</p><p>#{$#}</p>").hashEmUp
writeReport($allTestCaseResults)
end
end
end
The output it's generating is:
working one
working one
working one
It's not opening a browser or doing any of the subsequent code.
It looks like the code is having the problem mentioned in the Thread class documentation:
If we don't call thr.join before the main thread terminates, then all
other threads including thr will be killed.
Basically your main thread is finishing pretty instantaneously. However, the threads, which create browsers, take a lot longer than that. As result the threads get terminated before the browser opens.
By adding a long sleep at the end, you can see that your browsers can be opened by your code:
require 'watir-webdriver'
$chunkythread = ["A", "B", "C"]
$chunkythread.each do |z|
puts "pshaw"
Thread.new{
Thread.current["testwords"] = "ohai " + z.to_s
Thread.current["myBrowser"] = Watir::Browser.new :ff
puts Thread.current["testwords"] }
end
sleep(300)
However, for more reliability, you should join all the threads at the end:
require 'watir-webdriver'
threads = []
$chunkythread = ["A", "B", "C"]
$chunkythread.each do |z|
puts "pshaw"
threads << Thread.new{
Thread.current["testwords"] = "ohai " + z.to_s
Thread.current["myBrowser"] = Watir::Browser.new :ff
puts Thread.current["testwords"] }
end
threads.each { |thr| thr.join }
For the actual code example, putting #threads << Thread.current will not work. The join will be evaluating like #threads is empty. You could try doing the following:
$testCaseArray.each do |thisCase|
#threads << Thread.new {
puts "working one"
Thread.current["tBrowser"] = Watir::Browser.new :ff
# Do your other thread stuff
}
$alphabet[#count] = #threads.last
#count += 1
end
#threads.each do |thisThread|
thisThread.join
end
Note that I am not sure why you want to store the threads in $alphabet. I put in the $alphabet[#count] = #threads.last, but could be removed if not in use.
I uninstalled Watir 5.0.0 and installed Watir 4.0.2, and now it works fine.

Downloading binary files in Ruby

I am able to download my file, but I end up with a file that says it cannot be executed on my version of Windows. Here is my code:
Net::HTTP.start(url_base) do |http|
if response.code == '200'
self.size = response['content-length'].to_i
local_file = save_dir + File::SEPARATOR + strip_file_name(url_path)
if !application_exists?(local_file, response['content-length'].to_i) || overwrite_file?
build_display
self.progress_bar = ProgressBar.new(response['content-length'].to_i, PROGRESSBAR_SIZE_WIDTH)
File.open(local_file, 'w') { |f|
http.get(URI.escape(url_path)) do |str|
f.write str
#counter += str.length
#window.setpos(PROGRESSBAR_LOCATION_TOP, 1)
#window << (self.progress_bar.show(#counter)).center(WINDOW_WIDTH - 2)
#window.refresh
char = #window.getch
http.finish if char == 'c' # cancel download
end
}
end
end
end
If I download using open-uri:
file = open(url,
:content_length_proc => lambda { |content_length|
if content_length && 0 < content_length
self.progress_bar = ProgressBar.new(content_length, PROGRESSBAR_SIZE_WIDTH)
end
},
:progress_proc => lambda { |size|
##counter += str.length
#window.setpos(PROGRESSBAR_LOCATION_TOP, 1)
#window << (self.progress_bar.show(size)).center(WINDOW_WIDTH - 2)
#window.refresh
char = #window.getch
f.close if char == 'c' # cancel download
},
:read_timeout => 10)
FileUtils::cp(file, local_file)
Everything works....the problem is, I need to be able to cancel the download mid-stream. How can I a) download using net::http in a 'single-chunk' style, as I think open-uri does, or b) cancel a download mid-stream using open-uri?

Resources