Skip to content

WIP content-sqlite: preallocate space to avoid outside diskfull events #6217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chu11 wants to merge 11 commits into
base: master
Choose a base branch
from
Open

WIP content-sqlite: preallocate space to avoid outside diskfull events #6217

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bc19543
test: add 5m tmpfs
chu11 Aug 13, 2024
d6ce14b
content-sqlite: split truncate out of function
chu11 Aug 7, 2024
1f95a3d
content-sqlite: initialize variables on close db
chu11 Aug 13, 2024
ffa16c3
content-sqlite: open db given input parameters
chu11 Aug 14, 2024
cdbea4b
content-sqlite: support preallocate config
chu11 Aug 7, 2024
f366ae4
t: cover content-sqlite preallocate
chu11 Aug 13, 2024
a0e7c60
content-sqlite: disable journaling if hit ENOSPC
chu11 Aug 13, 2024
82eb65e
t: cover preallocate with journaling
chu11 Aug 15, 2024
816fae5
doc: document new content-sqlite config
chu11 Aug 15, 2024
297057f
fixup! content-sqlite: support preallocate config
chu11 Aug 20, 2024
059d388
test
chu11 Aug 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions src/modules/content-sqlite/content-sqlite.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -613,17 +613,21 @@ void stats_get_cb (flux_t *h,
json_decref (store_time);
}

static int content_sqlite_setup (struct content_sqlite *ctx, bool truncate)
{
if (truncate)
(void)unlink (ctx->dbfile);
return 0;
}

/* Open the database file ctx->dbfile and set up the database.
*/
static int content_sqlite_opendb (struct content_sqlite *ctx, bool truncate)
static int content_sqlite_opendb (struct content_sqlite *ctx)
{
int flags = SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE;
char s[128];
int count;

if (truncate)
(void)unlink (ctx->dbfile);

if (sqlite3_open_v2 (ctx->dbfile, &ctx->db, flags, NULL) != SQLITE_OK) {
log_sqlite_error (ctx, "opening %s", ctx->dbfile);
goto error;
Expand Down Expand Up @@ -936,7 +940,9 @@ int mod_main (flux_t *h, int argc, char **argv)
goto done;
if (process_args (ctx, argc, argv, &truncate) < 0)
goto done;
if (content_sqlite_opendb (ctx, truncate) < 0)
if (content_sqlite_setup (ctx, truncate) < 0)
goto done;
Comment on lines +1126 to +1127
Copy link
Member

@garlick garlick Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function name is not very descriptive. Maybe just do the

if (truncate)
    (void)unlink (ctx->dbfile)

inline?

if (content_sqlite_opendb (ctx) < 0)
goto done;
if (content_register_service (h, "content-backing") < 0)
goto done;
Expand Down