WebAPI: Support persisting WebUI client data

[Piccirello]

This PR adds two new endpoints for storing WebUI client data. These endpoints will allow certain data to be reused across all instances of the WebUI, regardless of browser or device. This data includes many of the preferences currently available in the Behavior tab of the Options dialog (color scheme, double click behavior, etc.), all of which are currently stored in browser local storage. This data will also include the Add Torrent dialog's default category and last used save path.

I do not expect this to be a drop-in replacement for everything currently stored in local storage, such as column widths and window sizes.

New endpoints:

• GET clientdata/load for retrieving current data/preferences
• POST clientdata/store for setting new data. Supplied data is merged server-side with existing data. Data can be deleted by setting its value to null.

[Piccirello]

@glassez I would especially like to get your opinion on this, as I know you've thought about this functionality in the past. Would this API be sufficient or does it need additional/different functionality?

[glassez]

@glassez I would especially like to get your opinion on this, as I know you've thought about this functionality in the past.

Let's not look into the implementation details for now, but rather discuss general issues.

First of all, it is worth recalling that I would expect to see some abstract storage of user data, rather than limiting its scope to preferences/settings. It's more a matter of terminology but it also determines the logic of its interface, which I suggest below.

As for its interface, I would provide three entrypoints:

1. setUserData: Accepts one or more key-value pairs and stores them in server-side storage.
2. getUserData: Accepts one or more keys and returns appropriate key-value pairs as JSON object.
3. removeUserData: Accepts one or more keys and removes associated data from server-side storage.

I believe that the above should cover the needs of saving client settings and states (and maybe allow something else in the future).

Note that if clients (i.e. client software) embed some identifier in the key names, this will allow them to avoid conflicts when accessing the same qBittorrent instance from different clients.

[Piccirello]

First of all, it is worth recalling that I would expect to see some abstract storage of user data, rather than limiting its scope to preferences/settings. It's more a matter of terminology but it also determines the logic of its interface, which I suggest below.

I don't personally foresee this being used for anything other than settings. In some cases one may consider a setting to be user data (e.g. a file path to save torrents to). Was there a specific example of user data that you had in mind?

As for its interface, I would provide three entrypoints:

1. setUserData: Accepts one or more key-value pairs and stores them in server-side storage.
2. getUserData: Accepts one or more keys and returns appropriate key-value pairs as JSON object.
3. removeUserData: Accepts one or more keys and removes associated data from server-side storage.

This is effectively the interface included in this PR, with the exception that removeUserData is currently combined with setUserData (you remove a key by setting its value to null).

Does setUserData always expect the entire data payload, or does it merge the payload with the existing user data? I believe you're stating the latter but it's good to be explicit here. setClientPreferences currently takes the former approach and setClientPreferencesOption takes the latter approach, but I'm leaning towards exclusively supporting the latter approach.

Note that if clients (i.e. client software) embed some identifier in the key names, this will allow them to avoid conflicts when accessing the same qBittorrent instance from different clients.

Now I'm really curious what you see these endpoints being used for. As I said, I see it being used to store WebUI-specific preferences, all of which would be global. Of course these endpoints would accept any generic key, so clients could implement the per-client identifiers you're describing, but it's hard for me to imagine without a concrete example use case.

[glassez]

I don't personally foresee this being used for anything other than settings. In some cases one may consider a setting to be user data (e.g. a file path to save torrents to). Was there a specific example of user data that you had in mind?

To begin with, even the client settings are just data for the server. (And, by the way, they should be stored in the appropriate place.)
And of course, I can hardly call settings, for example, "last used path", "last selected filter", and everything like that.

removeUserData is currently combined with setUserData (you remove a key by setting its value to null).

Well, it could make sense.

Does setUserData always expect the entire data payload, or does it merge the payload with the existing user data?

If it wasn't clear from the previous comment, then by "user data" I mean the single piece of data referenced by its key/name. setUserData just writes the data it receives to the storage and doesn't touch anything else. Moreover, it is intended that some particular client may not know (and is not required to know) about all the data stored on the server.

Now I'm really curious what you see these endpoints being used for. As I said, I see it being used to store WebUI-specific preferences, all of which would be global.

They are intended to be used to store any data that makes sense for the client application to store between launches, i.e. client-specific settings, the states (sizes, positions, etc.) of various widgets, the history of data entered in some controls, etc.

Of course these endpoints would accept any generic key, so clients could implement the per-client identifiers you're describing, but it's hard for me to imagine without a concrete example use case.

After all, there may (and do) exist different alternative clients. So, if someone were to use several different clients at the same time (alternately), then they might have conflicts of some data having commonly used keys (well, for example, AddTorrentDialog/Size), so it makes sense to use the client ID, for example, qBittorrent/AddTorrentDialog/Size, VueTorrent/AddTorrentDialog/Size).

[Piccirello]

I've updated the code and the PR description to reflect these changes.

They are intended to be used to store any data that makes sense for the client application to store between launches, i.e. client-specific settings, the states (sizes, positions, etc.) of various widgets, the history of data entered in some controls, etc.

I still expect some of this data to be unique per-client, such as sizes and positions, since the WebUI can be opened on many different devices with different screen sizes and resolutions. I think we agree on the broader points though.

[glassez]

I still expect some of this data to be unique per-client, such as sizes and positions, since the WebUI can be opened on many different devices with different screen sizes and resolutions.

I wrote about this in the last paragraph of the previous comment. Or do you mean something else?

[glassez]

@Piccirello
It seems that you still didn't understand the concept I explained, which is as follows:

2. getUserData: Accepts one or more keys and returns appropriate key-value pairs as JSON object.

Moreover, it is intended that some particular client may not know (and is not required to know) about all the data stored on the server.

You should never return all the stored data but only explicitly requested by client.

[glassez]

As for setUserData, wouldn't it be easier to accept the input data as separate parameters rather than as a single parameter containing JSON?

[glassez]

@Chocobo1
What do you think about this feature in general?

[Ryu481]

Will this also be used by the GUI? Since some of the settings in the WebUI are also present in the GUI client.

[Piccirello]

I wrote about this in the last paragraph of the previous comment. Or do you mean something else?

When I say client I mean each web browser that access the WebUI. If I'm using two different computers, or a computer and a mobile device, I don't necessarily want them to have the same window sizes. This is very different than the GUI where you want to persist all of these settings because the GUI is only ever accessed from a single device.

You should never return all the stored data but only explicitly requested by client.

Ah I did miss that. I generally agree that's useful though I also think it would be useful to fetch all data, that way it can be cached in the client (like we do with preferences today).

As for setUserData, wouldn't it be easier to accept the input data as separate parameters rather than as a single parameter containing JSON?

The current approach is easier as it means that values can be any valid json value. It also means we can send up one json blob. Our server doesn't yet support sending payloads with a content-type of application/json so this is an easy workaround.

Will this also be used by the GUI? Since some of the settings in the WebUI are also present in the GUI client.

Please see the PR description. This is only for settings unique to the WebUI, most of which are currently persisted in browser local storage.

[Ryu481]

I was thinking about tracker icons and last used save path. These settings currently exist in the gui but could potentially be added to the webUI.

[Piccirello]

I was thinking about tracker icons and last used save path. These settings currently exist in the gui but could potentially be added to the webUI.

Oh I misunderstood. Indeed those settings which are otherwise considered gui-only could use this. Last used save path is actually called out in the PR description and was one of the motivators for building this.

[glassez]

When I say client I mean each web browser that access the WebUI. If I'm using two different computers, or a computer and a mobile device, I don't necessarily want them to have the same window sizes.

Well, this can also be solved by using appropriate (more detailed) data identification. Or do you have a different idea?

I also think it would be useful to fetch all data, that way it can be cached in the client (like we do with preferences today).

This would fetch and cache unrelated data (related to another client software, browser etc.) as well. So I doubt it is a good idea.
Moreover, even the related data may not be required all at once, and some may not be needed at all in a given session (if certain parts of the UI are not used by the user). So they can be loaded on demand and cached if necessary.

[Piccirello]

Well, this can also be solved by using appropriate (more detailed) data identification. Or do you have a different idea?

This new API is for settings/data that will be shared among different instances of the WebUI (i.e. across different browsers/computers). The data you're describing is better left in local storage as it exists today. We don't really get any benefit to persisting it server-side if it's only ever going to be used by a single instance of the WebUI. That's also why I initially called this "settings".

This would fetch and cache unrelated data (related to another client software, browser etc.) as well. So I doubt it is a good idea. Moreover, even the related data may not be required all at once, and some may not be needed at all in a given session (if certain parts of the UI are not used by the user). So they can be loaded on demand and cached if necessary.

My comment above also addresses this, but the data that this will be storing should't ever be "unrelated" due to a different browser. That data will continue to live in local storage. The data stored by the API should be relevant to every instance of the WebUI that's loaded.

I don't think it's worth spending time on the use case of this somehow conflicting wtith alternative WebUIs because you can only run one WebUI at a time - either the official WebUI or exactly one alternative WebUI.

[Piccirello]

Feedback has been addressed. I've moved the WebUI changes out of this PR and into #23191.

[Piccirello]

^ rebased onto master, no changes to this PR

[Piccirello]

Rebased onto master to fix merge conflict

[Piccirello]

Ready to merge?

[glassez]

@Piccirello
Thank you.