У меня есть длительная задача rake, которая каждую ночь обновляет данные MLS. Задача сначала обновляет данные списка, затем, после обновления всех списков, изображения загружаются через активное хранилище Rails 5.2. Списки обновляются нормально в течение 2-4 часов, но загрузка изображения будет длиться от 10 до 30 минут, прежде чем произойдет сбой с той же ошибкой Rets::InvalidIdentifier: Got error code 20402 (Invalid Identifier).
Эта ошибка исходит от сервера MLS Rets.
Я использую гем estately/rets и Suckerpunch для асинхронного запуска задачи. Код, который приводит к ошибке:
line 156 puts 'Now save the pictures'
157
158 ids.each do |id|
159 p = Listing.find(id)
160 mlsid = p.mlsid
161 photos = client.objects '*', {
162 resource: 'Property',
object_type: 'Photo',
resource_id: mlsid
}
if photos.count < 1
next
end
photos.each_with_index do |photo, index|
p.images.attach(io: StringIO.new(photo.body), filename: "#{p.mlsid}-#{index}.jpg")
end
puts "#{photos.count} images attached to listing_id = #{p.id}"
end # pictures ids.each do
client.logout
и полная трассировка ошибки ниже:
I, [2018-06-07T06:26:53.943065 #4] INFO -- : S3 Storage (36.8ms) Uploaded file to key: XWoiiTfSh9LR25zKA8y4tpc8 (checksum: oybNBOCMwYTKqaTO5j5Nuw==)
D, [2018-06-07T06:26:53.944271 #4] DEBUG -- : (1.0ms) BEGIN
D, [2018-06-07T06:26:53.946751 #4] DEBUG -- : ActiveStorage::Blob Create (1.4ms) INSERT INTO "active_storage_blobs" ("key", "filename", "content_type", "metadata", "byte_size", "checksum", "created_at") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["key", "XWoiiTfSh9LR25zKA8y4tpc8"], ["filename", "09973750-21.jpg"], ["content_type", "image/jpeg"], ["metadata", "{\"identified\":true}"], ["byte_size", 35097], ["checksum", "oybNBOCMwYTKqaTO5j5Nuw=="], ["created_at", "2018-06-07 06:26:53.944514"]]
D, [2018-06-07T06:26:53.950865 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] (1.7ms) BEGIN
D, [2018-06-07T06:26:53.952073 #4] DEBUG -- : (4.7ms) COMMIT
D, [2018-06-07T06:26:53.953379 #4] DEBUG -- : (1.0ms) BEGIN
D, [2018-06-07T06:26:53.953591 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] ActiveStorage::Blob Update (1.6ms) UPDATE "active_storage_blobs" SET "metadata" = $1 WHERE "active_storage_blobs"."id" = $2 [["metadata", "{\"identified\":true,\"width\":180,\"height\":225,\"analyzed\":true}"], ["id", 25654]]
D, [2018-06-07T06:26:53.956386 #4] DEBUG -- : ActiveStorage::Attachment Create (0.9ms) INSERT INTO "active_storage_attachments" ("name", "record_type", "record_id", "blob_id", "created_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["name", "images"], ["record_type", "Listing"], ["record_id", 441544], ["blob_id", 25655], ["created_at", "2018-06-07 06:26:53.954865"]]
D, [2018-06-07T06:26:53.958520 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] (2.7ms) COMMIT
D, [2018-06-07T06:26:53.958787 #4] DEBUG -- : Listing Update (1.3ms) UPDATE "listings" SET "updated_at" = $1 WHERE "listings"."id" = $2 [["updated_at", "2018-06-07 06:26:53.956749"], ["id", 441544]]
I, [2018-06-07T06:26:53.959030 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] Performed ActiveStorage::AnalyzeJob (Job ID: e18de7a4-86bb-4075-844d-5466801613d0) from Async(default) in 47.95ms
D, [2018-06-07T06:26:53.960650 #4] DEBUG -- : (1.4ms) COMMIT
I, [2018-06-07T06:26:53.961439 #4] INFO -- : [ActiveJob] Enqueued ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) to Async(default) with arguments: #<GlobalID:0x0000000003645a20 @uri=#<URI::GID gid://schoolsparrow/ActiveStorage::Blob/25655>>
22 images attached to listing_id = 441544
D, [2018-06-07T06:26:53.964406 #4] DEBUG -- : Listing Load (1.4ms) SELECT "listings".* FROM "listings" WHERE "listings"."id" = $1 LIMIT $2 [["id", 441545], ["LIMIT", 1]]
D, [2018-06-07T06:26:53.965427 #4] DEBUG -- : ActiveStorage::Blob Load (1.1ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 LIMIT $2 [["id", 25655], ["LIMIT", 1]]
I, [2018-06-07T06:26:53.966994 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] Performing ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) from Async(default) with arguments: #<GlobalID:0x00000000035480a0 @uri=#<URI::GID gid://schoolsparrow/ActiveStorage::Blob/25655>>
D, [2018-06-07T06:26:54.003064 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] (0.9ms) BEGIN
D, [2018-06-07T06:26:54.005782 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] ActiveStorage::Blob Update (1.3ms) UPDATE "active_storage_blobs" SET "metadata" = $1 WHERE "active_storage_blobs"."id" = $2 [["metadata", "{\"identified\":true,\"width\":202,\"height\":225,\"analyzed\":true}"], ["id", 25655]]
D, [2018-06-07T06:26:54.012433 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] (6.0ms) COMMIT
I, [2018-06-07T06:26:54.012766 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] Performed ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) from Async(default) in 45.53ms
rake aborted!
Rets::InvalidIdentifier: Got error code 20402 (Invalid Identifier)
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/error_checker.rb:33:in `check'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/multipart.rb:35:in `check_for_invalids_parts!'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/multipart.rb:29:in `parse'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/client.rb:216:in `create_parts_from_response'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/client.rb:203:in `objects'
/app/app/jobs/mred_job.rb:162:in `block in perform'
/app/app/jobs/mred_job.rb:159:in `each'
/app/app/jobs/mred_job.rb:159:in `perform'
/app/lib/tasks/update.rake:10:in `update_mred'
/app/lib/tasks/update.rake:6:in `block (2 levels) in <top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in `load'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:27:in `run'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:360:in `exec'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor.rb:369:in `dispatch'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:20:in `dispatch'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/base.rb:444:in `start'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:10:in `start'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/exe/bundle:30:in `block in <top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/exe/bundle:22:in `<top (required)>'
/app/bin/bundle:3:in `load'
/app/bin/bundle:3:in `<main>'
Tasks: TOP => update:mred
Кто-нибудь знает, почему это работает какое-то время, а затем вылетает с той же ошибкой? Любые мысли будут высоко оценены!
