Support for AWS S3 backend?

[stockersky]

I just gave a try to django-unused-media today.
It works perfectly out of the box with my media files hosted on my own workstation.

But, in production, I use AWS S3 to store media and static files into a bucket.
And I got this exception : NotImplementedError: This backend doesn't support absolute paths.

Is AWS S3 supported?
Or I am missing something?

Thanks!

[mrkn16ht]

Hello, akolpakov, it works perfectly in root storages.
But same with stockersky, I changed to AWS S3 to store media and static files into a bucket.
And got the same problem
NotImplementedError: This backend doesn't support absolute paths.

Here is the traceback:

Traceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django_unused_media/management/commands/cleanup_unused_media.py", line 71, in handle
    unused_media = get_unused_media(options.get('exclude') or [])
  File "/usr/local/lib/python3.6/site-packages/django_unused_media/cleanup.py", line 75, in get_unused_media
    used_media = get_used_media()
  File "/usr/local/lib/python3.6/site-packages/django_unused_media/cleanup.py", line 35, in get_used_media
    append_if_not_exists(media, storage.path(value))
  File "/usr/local/lib/python3.6/site-packages/django/core/files/storage.py", line 106, in path
    raise NotImplementedError("This backend doesn't support absolute paths.")

Could you please help, thanks!

[akolpakov]

Only local storages are supported. If anyone is ready to contribute - you are welcome!

[rarenatoe]

I am getting an issue where an image gets replaced in s3, but for whatever reason when my backend retrieves it it randomly gets a picture between the previously used ones and the latest one as if none of them had been erased and this picture gets updated every 15 minutes or so...
I am very clueless on what is causing it.
By the looks of it, my django using boto3 is not keeping a local storage... or is it?
I'm not asking to fix this error (I'm getting it too..) but did you @stockersky or @mrkn16ht find any solution?

[fsecada01]

Seems like the issue is that the module relies on django.core.files.storage.path to commit objects to a set of media instances, whereas cloud storage objects can only be found via URLs. The solution may rest with adding a url=True argument to the get_media functions, and then use django.core.files.storage.url to add to the media set object. I just made a fork and experimenting with this now.

[fsecada01]

So after spending the day on this, I can't find a feasible way of deleting media files without authentication for the cloud storage backend.

I think the best way forward would be to write some integration with django-storages.

[patroqueeet]

Hey, glad this lib exists.

I have a mixed setup. local/s3. can we just exclude s3/django-storages at least?

that would be very nice...

[mfxa]

There's a working solution to NotImplementedError: This backend doesn't support absolute paths. here, at least when it comes to django-extensions' unreferenced_files:
https://stackoverflow.com/a/48785695

[SupraSummus]

I've implemented handling of remote storages in my fork:

• EE@b08779f
• EE@0235c07

You can pick working version from master branch at my fork. It has some other tweaks, not only remote storage handling.