Increase integer column types to bigint, fix mysql index errors

[ffried]

The integer type in non-SQLite databases is too small to hold Date.now() values.
This increases the type to bigint. To deal with existing databases, alter table commands were added. They should simply succeed in case the type already matches (which is the case for newly created databases).
This fixes #136.

Also, in MySQL you can't use text with indices (unless specifying a prefix length). This is why I've changed the data type to varchar(4096) for these columns. Note that no alter table is needed for these, because the creation of the table already fails. The size (4096) is currently arbitrarily chosen and can be set to anything large enough (up to 65,535).

[Copilot]

Pull Request Overview

This PR addresses database compatibility issues by increasing integer column types to bigint (to properly store Date.now() values) and changing indexed text columns to varchar(4096) for MySQL compatibility. It also adds migration logic for existing databases and refactors the periodic cleanup into a separate function.

Key Changes:

• Changed integer columns storing timestamps to bigint across all tables
• Introduced MySQL-compatible varchar(4096) for indexed text columns
• Added migration logic with changeIntToBigInt() function to update existing databases

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tiagozip]

thanks, ill start reviewing and testing later. just to be sure, can you confirm that varchar(4096) does indeed work on both SQLite, postgres and MySQL?

[ffried]

It's only used for MySQL. Postgres an sqlite continue to use text.

[ancashoria]

I have tested this PR and can confirm it fixes this bug

[ancashoria]

After more trial and error, it seems there are still issues with Postgres compatibility

[ffried]

After more trial and error, it seems there are still issues with Postgres compatibility

This might totally be the case, since I've only fixed the most obvious once (namely the ones storing timestamps).
Can you elaborate on which issues are still present?

[ancashoria]

One issue I encountered was on the redeem request where I got column reference "count" is ambiguous which sounds like an sql error to me.

I also noticed that the UI graph for a site I triggered a few solve requests was not displaying anything, although the count was counting them.

Lastly, I think this SQL query is not working because I got "Invalid site key or secret" when atempting a siteverify. I double checked my secret and all looks good.

[ffried]

@ancashoria

• The count issue is obvious but also simple to fix (fully qualifying works the same no all DBs). I've changed that.
• The brackets in the select you referenced shouldn't be needed anyways - maybe that fixes it? I don't see any other issues with that select otherwise. Also, this is not a siteverify - this is the challenge request. The error message is also incorrect there, since there's no secret involved. I've adjusted that.
• As for the graphs, I'm not sure where the issue stems from. Have you tried this with e.g. SQLite?

Can you retest with the changes and let me know if they work for you?

[ancashoria]

Sure, I'll give it a shot.

[ffried]

@ancashoria Any update?

[ancashoria]

I have switched to SQLite so I won't be testing this any time soon. Sorry

[ffried]

I've tested this now with PostgreSQL and it's working just fine. We're running our instance now using the current HEAD of this PR.

[tiagozip]

thanks! could you confirm if you have tested this on mysql as well?

[ffried]

@tiagozip I don't have a mysql setup right now. I've tested the migrations on mysql (which is where the need for the fix for the index on text columns was discovered). Other than that, I don't expect mysql to have any issues with the changes I've made here.
Best case: Mysql now works flawlessly as well.
Worst case: It's still broken (as before).

I can't guarantee that I'll find the time soon to test MySQL. I was able to test Postgres using our (now) production setup, that I could easily use for testing since there was nothing in it, but the environment was there.

[ffried]

@tiagozip Alright, I've spent a couple more hours on this and tested it with mysql. My dislike for MySQL grew even more. The changes needed for Postgres are simply due to it being properly typed (compared to SQLite). That's fine and expected. MySQL, however, now needs different column types (due to index limitations) and escaping all table names (which I think should be handled by Bun...?).
But now it works.

Also note that Bun seems to have difficulties with longer MySQL passwords: oven-sh/bun#26195

Besides that, I've found a couple of other issues. Though that's not related to this PR and I'll open separate issues for that.

[ffried]

Forgot to test the redeem. That works now as well...