Skip to content

Commit

Permalink
feat: adds some threads
Browse files Browse the repository at this point in the history
* adds in faraday-retry
* adds a basic threadpool
  • Loading branch information
niquerio committed Sep 30, 2022
1 parent 95684bd commit 8ccb5d6
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 7 deletions.
1 change: 1 addition & 0 deletions .env-example
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ WEBLOGIN_SECRET=your-weblogin-secret
WEBLOGIN_ID=your-weblogin-id'
WEBLOGIN_ON=true
CATALOG_SOLR=http://url-to-solr/solr/biblio
MAX_THREADS=5
4 changes: 4 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ GEM
byebug (11.1.3)
coderay (1.1.3)
concurrent-ruby (1.1.10)
concurrent-ruby-ext (1.1.10)
concurrent-ruby (= 1.1.10)
crack (0.4.5)
rexml
diff-lcs (1.5.0)
Expand Down Expand Up @@ -167,6 +169,8 @@ PLATFORMS

DEPENDENCIES
byebug
concurrent-ruby
concurrent-ruby-ext
faraday
faraday-retry
net-smtp
Expand Down
23 changes: 16 additions & 7 deletions lib/report_generator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,31 @@ class ReportGenerator
def initialize(barcodes: [])
@barcodes = barcodes
@umich_catalog_items = UmichCatalogItems.for(barcodes: @barcodes)
Logger.new($stdout).info("fetched umich catalog items")
end

def run
queue = Queue.new
queue.push(header_row)

@barcodes.each do |barcode|
umich_item = @umich_catalog_items.item_for_barcode(barcode)
next if umich_item.nil?
worldcat_summary = WorldCatSummary.for(umich_item.oclc || [])
barcode_queue = @barcodes.inject(Queue.new, :push)

queue.push(
format_line(umich_item: umich_item, worldcat_summary: worldcat_summary)
)
threads = Array.new(ENV.fetch(MAX_THREADS)) do
Thread.new do
until barcode_queue.empty?
barcode = barcode_queue.shift
umich_item = @umich_catalog_items.item_for_barcode(barcode)
next if umich_item.nil?
worldcat_summary = WorldCatSummary.for(umich_item.oclc || [])

queue.push(
format_line(umich_item: umich_item, worldcat_summary: worldcat_summary)
)
end
end
end

threads.each(&:join)
print(queue)
end

Expand Down

0 comments on commit 8ccb5d6

Please sign in to comment.