Ошибка 500 при загрузке файлов с Filer на S3 с использованием boto3 и django-хранилищ на Elastic Beanstalk

Я получаю следующую ошибку 500 http при попытке загрузить файл через администратора django-cms Filer в моем приложении, развернутом в Elastic Beanstalk:

В консоли браузера от fileuploader.min.js:26:

POST http://app-staging.us-east-1.elasticbeanstalk.com/admin/filer/clipboard/operations/upload/1/?qqfile=example.jpg 500 (Internal Server Error)

Я использую django-cms (и, следовательно, easy-thumbnails), django-storages и boto3. При развертывании команда collectstatic в моем конфигурационном файле .ebextensions отлично работает, и все файлы прекрасно хранятся в корзине S3 в /staging/static/.

Удаление назначения DEFAULT_FILE_STORAGE работает нормально, но очевидно, что это неприемлемо, поскольку каждое eb deploy любых изменений в beanstalk приводит к созданию нового экземпляра и стиранию предыдущих загрузок файлов.

Трассировки стека:

Internal Server Error: /admin/filer/clipboard/operations/upload/1/
 Traceback (most recent call last):
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
     response = get_response(request)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
     response = self._get_response(request)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
     response = self.process_exception_by_middleware(e, request)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
     response = wrapped_callback(request, *callback_args, **callback_kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
     return view_func(*args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/filer/admin/clipboardadmin.py", line 114, in ajax_upload
     file_obj.save()
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/filer/models/imagemodels.py", line 53, in save
     super(Image, self).save(*args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/filer/models/abstract.py", line 74, in save
     super(BaseImage, self).save(*args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/filer/models/filemodels.py", line 194, in save
     super(File, self).save(*args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/polymorphic/models.py", line 74, in save
     return super(PolymorphicModel, self).save(*args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/base.py", line 806, in save
     force_update=force_update, update_fields=update_fields)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/base.py", line 835, in save_base
     self._save_parents(cls, using, update_fields)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/base.py", line 862, in _save_parents
     self._save_table(cls=parent, using=using, update_fields=update_fields)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/base.py", line 922, in _save_table
     result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/base.py", line 961, in _do_insert
     using=using, raw=raw)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
     return getattr(self.get_queryset(), name)(*args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/query.py", line 1060, in _insert
     return query.get_compiler(using=using).execute_sql(return_id)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1098, in execute_sql
     for sql, params in self.as_sql():
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1051, in as_sql
     for obj in self.query.objs
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1051, in <listcomp>
     for obj in self.query.objs
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1050, in <listcomp>
     [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1000, in pre_save_val
     return field.pre_save(obj, add=True)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/fields/files.py", line 297, in pre_save
     file.save(file.name, file.file, save=False)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/filer/fields/multistorage_file.py", line 122, in save
     super(MultiStorageFieldFile, self).save(name, content, save)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/easy_thumbnails/files.py", line 670, in save
     super(ThumbnailerFieldFile, self).save(name, content, *args, **kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/db/models/fields/files.py", line 95, in save
     self.name = self.storage.save(name, content, max_length=self.field.max_length)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/django/core/files/storage.py", line 54, in save
     return self._save(name, content)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 452, in _save
     self._save_content(obj, content, parameters=parameters)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 467, in _save_content
     obj.upload_fileobj(content, ExtraArgs=put_parameters)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/boto3/s3/inject.py", line 513, in object_upload_fileobj
     ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
     return future.result()
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/s3transfer/futures.py", line 73, in result
     return self._coordinator.result()
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/s3transfer/futures.py", line 233, in result
     raise self._exception
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/s3transfer/tasks.py", line 126, in __call__
     return self._execute_main(kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/s3transfer/tasks.py", line 150, in _execute_main
     return_value = self._main(**kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/s3transfer/upload.py", line 685, in _main
     client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/botocore/client.py", line 317, in _api_call
     return self._make_api_call(operation_name, kwargs)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/botocore/client.py", line 589, in _make_api_call
     api_params, operation_model, context=request_context)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/botocore/client.py", line 624, in _convert_to_request_dict
     api_params, operation_model)
   File "/opt/python/run/venv/local/lib/python3.6/site-packages/botocore/validate.py", line 291, in serialize_to_request
     raise ParamValidationError(report=report.generate_report())
 botocore.exceptions.ParamValidationError: Parameter validation failed:
 Invalid type for parameter ContentType, value: None, type: <class 'NoneType'>, valid types: <class 'str'>

settings.py

STATIC_ROOT = os.path.join(BASE_DIR, '..', 'www', 'static')
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'static', 'eatingveg'),
)

MEDIA_ROOT = os.path.join(BASE_DIR, '..', 'www', 'media')

# AWS S3 Settings
AWS_STORAGE_BUCKET_NAME = os.environ.get('EV_S3_BUCKET_NAME')
AWS_S3_REGION_NAME = os.environ.get('EV_S3_REGION_NAME')  # e.g. us-east-2
AWS_ACCESS_KEY_ID = os.environ.get('EV_AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('EV_AWS_SECRET')
AWS_QUERYSTRING_AUTH = False
AWS_IS_GZIPPED = True

# Tell django-storages the domain to use to refer to static files.
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = "https://%s/staging/" % AWS_S3_CUSTOM_DOMAIN
MEDIA_URL = "https://%s/staging/media/" % AWS_S3_CUSTOM_DOMAIN

# Tell the staticfiles app to use S3Boto3 (via our StaticStorage subclass)
# storage when writing the collected static files
# (when you run `collectstatic`).
STATICFILES_LOCATION = 'staging/static'
STATICFILES_STORAGE = 'custom_storages.StaticStorage'

MEDIAFILES_LOCATION = 'staging/media'
DEFAULT_FILE_STORAGE = 'custom_storages.MediaStorage'

custom_storages.py

# custom_storages.py
from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage


class StaticStorage(S3Boto3Storage):
    location = settings.STATICFILES_LOCATION


class MediaStorage(S3Boto3Storage):
    location = settings.MEDIAFILES_LOCATION

person Dluks    schedule 08.02.2018    source источник


Ответы (2)


Оказывается, проблема была с django-storages. При добавлении файла через администратора Filer он вызывал boto с пустым ContentType, что boto не нравится, как задокументировано здесь. запрос на вытягивание, разрешающий это, разрешив None, был отправлен, но еще не объединен .

Разветвив bxm156, я смог решить проблему.

person Dluks    schedule 08.02.2018

Обновите django-storages до версии 1.6.6. Это решает проблему.

person somecallitblues    schedule 11.04.2018