Data Library

The goal of the Data Library is to provide a suite of layers which are managed by the SeaSketch team and can be easily added to any project. Making a map portal should be super easy in SeaSketch. The data should either be:

1. Already available in SeaSketch
2. Easy to add from a data service, or
3. Easy to upload and style

The Data Library tackles the first part of that formula.

Architecture

Data Library entries can take many forms. There is a taxonomy of these items, and each is going to be implemented a little differently.

1. Entries which are a single, or group of multiple, table of contents items. (Coral Reef Watch being the first)
2. Entries which are a basemap (Global Fishing Watch)

Table of Contents Items

In the case of our first use-case, NOAA Coral Reef Watch provides a set of layers as NetCDF files (+ a kml) which are updated on a web directory approximately daily. The general pattern supporting this Data Library item is as follows.

1. A folder, containing table of contents items representing each of these layers is manually created in the superuser project. This includes creating the initial cartography, metadata, attribution, and interactivity.
2. Manually in the database, table_of_contents_items.data_library_template_id is set on each item with a unique identifier, such as "CRW_ROOT" for the root folder, and "CRW_BAA" for the BAA (bleaching alert level) layer.
3. An update job is created as a graphile_worker task which runs like a cron job when setup in the server configuration. These tasks take advantage of a common set of utilities for writing data library update tasks, and store metadata in table_of_contents_items.data_library_metadata to keep track of versioning and update dates between what's on SeaSketch and what is on the authoritative source's website. These utilities use the "replace data source" workflow and lambda already used to support data upload versioning in order to update data library templates. Code in the replace_data_source stored procedure identifies project copies of Data Library items, and also keeps them up to date when the template source changes.
4. Once there are ID'd layers in the superuser project, and infrastructure to support keeping them up to date, they can be added to the client in DataLibraryModal.tsx, referencing the root template ID to be copied into the destination project.

Assigning Data Library Template IDs

Both the data library and the geography system rely on template layers (really, Table of Contents Items). New data library items and the geography system require manual setup in the superuser project to create these layers and identify them. For example, new installations of SeaSketch require a MARINE_REGIONS_EEZ_LAND_JOINED layer in order to support clipping to Exclusive Economic Zones. In order to set this layer up, perform the following steps:

1. Create a table of contents item representing this layer in the superuser project by uploading a spatial data file
2. Assign appropriate symbology, popups, metadata, etc. These will all carry over to projects copying the template
3. Within the ToC editor's settings tab, copy the stable_id from the bottom of the form (e.g. GD2Ir0uB1)
4. From the postgres shell, execute the following function (replacing the stable_id and template id with appropriate values): select assign_data_library_template_id('Q8B_pTEnq', 'MARINE_REGIONS_EEZ_LAND_JOINED');