`source` attribute of file and dir entities with special characters

[LauLauThom]

Dear all,

I am opening this issue to discuss the source attribute of Data entities.
It is related to the escaping of special characters, which has been the topic of some past issues and PR (#217 #225).
The fixes are not yet part of the current release at the time of writing this (0.13.0), so please use the latest version "from source" to test the example below.

So I noticed that the source attribute of File and Dataset entities currently return a Path object.
This Path is valid (i.e resolve to the actual file or dir on disk) when the filepath does not contain special characters (like spaces or accentuated chars).
With special characters however, the returned Path contains the escaped special characters (as in a URL), and so source.exists() returns false.
The workaround to retrieve the actual filepath as a "valid" Path, is to unquote with Path(unquote(str(file.source))), which is quite heavy notation.

I know that the source can also be a URL, in which case it makes sense to keep the % chars.
Yet, I see a potential for hard-to-debug issues, when someone would write the code expecting source to return a valid path, to later find out that the code does not work as soon as the filepath contains special characters.

I was wondering if source could return a valid path when it's pointing to a local entity, and a URL when it's a remote entity.
If source is a property in the class FileOrDir, the getter could be something like

# in rocrate/model/file_or_dir.py
import validators

class FileOrDir(DataEntity):
    
    def get_source(self):

        if validators.url(self.source): # a URL to remote data, then return as such
            return self.source
        
        else:
            return Path(unquote(str(self.source))) # return as a "valid" Path 

Below is a short example demonstrating the current behaviour.

from rocrate.rocrate import ROCrate
import os
from urllib.parse import unquote
from pathlib import Path

new_crate_root = "./test_crate"

# create a directory with a single text file
if not os.path.exists(new_crate_root):
    os.mkdir(new_crate_root)

# create a dummy file, illustrating "usual" behaviour
with open(os.path.join(new_crate_root, "file.txt"), "w") as f:
    f.write("empty")

# create a file with spaces in the name
with open(os.path.join(new_crate_root, "file with space.txt"), "w") as f:
    f.write("empty")

# Initialize a crate from the directory
crate = ROCrate(new_crate_root, init=True)
crate.write(new_crate_root) 

# then reload it, this time parsing it
crate = ROCrate(new_crate_root)
list_files = crate.get_by_type("File")

for file_entity in list_files : 
    print(f"{file_entity.source=}") 
    print("source exists :", file_entity.source.exists())

    source_unquoted = Path(unquote(str(file_entity.source)))
    print("source (escaped) exists :", source_unquoted.exists() )
    print("")

outcome :

file_entity.source=PosixPath('test_crate/file.txt')
source exists : True
source (escaped) exists : True

file_entity.source=PosixPath('test_crate/file%20with%20space.txt')
source exists : False
source (escaped) exists : True

[simleo]

You cannot simply unquote self.source unconditionally after just checking that it's not a URL.

First of all, if you take a look at the code in file.py and dataset.py, self.source can be None, a BytesIO or StringIO, a str or a pathlib.Path.

Secondly, files on disk can contain a % in their name. For instance, a%20b.txt is a valid path. This means that unquoting would lead to a nonexistent a b.txt. In fact, while investigating this, I found out that the current code is already making this mistake!

>>> from rocrate.rocrate import ROCrate
>>> crate = ROCrate()
>>> f = crate.add_file("/tmp/a%20b.txt")
>>> f.id
'a%2520b.txt'
>>> f.source
'/tmp/a%20b.txt'
>>> f.id
'a%2520b.txt'
>>> crate.write("/tmp/crate")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/simleo/git/ro-crate-py/rocrate/rocrate.py", line 476, in write
    writable_entity.write(base_path)
  File "/home/simleo/git/ro-crate-py/rocrate/model/file.py", line 82, in write
    self._copy_file(self.source, out_file_path)
  File "/home/simleo/git/ro-crate-py/rocrate/model/file.py", line 70, in _copy_file
    shutil.copy(path, out_file_path)
  File "/usr/lib/python3.12/shutil.py", line 435, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.12/shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/a b.txt'