Skip to content

Conversation

@ziggie1984
Copy link
Collaborator

@ziggie1984 ziggie1984 commented Oct 13, 2025

Depends on #10287

This PR implements most of the Insert and Delete methods we need for the unit tests in the following PR.

@ziggie1984 ziggie1984 changed the base branch from master to elle-payment-sql-series-new October 13, 2025 19:13
@ziggie1984 ziggie1984 self-assigned this Oct 13, 2025
@ziggie1984 ziggie1984 added payments Related to invoices/payments sql labels Oct 13, 2025
@ziggie1984 ziggie1984 added this to v0.21 Oct 13, 2025
@ziggie1984 ziggie1984 added this to the v0.21.0 milestone Oct 13, 2025
@saubyk saubyk moved this to In progress in v0.21 Oct 14, 2025
@ziggie1984 ziggie1984 changed the title [Part 3|*] Implement FetchPayment for SQL Backend [Part 3|*] Implement Second Part for SQL Backend functions Oct 15, 2025
@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch 2 times, most recently from 80de0dd to c0134ed Compare October 15, 2025 15:12
@ziggie1984 ziggie1984 marked this pull request as ready for review October 15, 2025 15:13
@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch 3 times, most recently from f0e3302 to 99ddf07 Compare October 17, 2025 07:53
@ziggie1984
Copy link
Collaborator Author

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a significant portion of the insert and delete methods for the new SQL backend for the payments database. The changes are extensive and include new SQL queries, data conversion functions, and the implementation of several methods on the SQLStore.

The use of ExecuteBatchQuery and ExecutePaginatedQuery is great for performance. The new PaymentAndIntent interface is a good abstraction.

I've found a few areas for improvement:

  • Adherence to the function commenting style guide for exported functions.
  • Consistent use of context.Context propagation instead of context.TODO().
  • Opportunities for performance improvements by using batch inserts and optimizing queries to avoid extra fetches.
  • A potential code smell related to handling of nil slices from the database.

Overall, this is a solid step forward in the SQL backend implementation. Addressing these points will improve the code's maintainability, robustness, and performance.

Comment on lines 281 to 286
// TODO(ziggie): Can we get rid of this?
// This has to be in place otherwise the
// comparison will not match.
if record.Value == nil {
record.Value = []byte{}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The TODO here highlights a potential code smell. This workaround of converting a nil slice to an empty slice might be hiding an issue or could be simplified. It would be beneficial to investigate whether the downstream code that consumes these records can be made to handle nil slices gracefully. This would remove the need for this conversion and make the data handling more consistent.

Comment on lines +739 to +1058
func (s *SQLStore) insertRouteHops(ctx context.Context, db SQLQueries,
hops []*route.Hop, attemptID uint64) error {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function inserts route hops and their associated data one by one in a loop. For routes with many hops, this can result in a large number of database queries, impacting performance. Consider refactoring this to use batch inserts. sqlc supports batch inserts, which would require modifying the SQL queries to accept slices of parameters and then calling them once with all the data for the hops.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gemini-code-assist a batch insert approach is not so easy to keep compatibility with the postgres and sqlite backend.

For typical payment routes (2-10 hops), the current approach is reasonable because:

  1. It's simple and maintainable
  2. Already runs in a transaction (provides batching)
  3. The complexity of multi-row INSERT may not be worth it for small batches
  4. Each hop has different optional data (MPP, AMP, blinded), making batch insert complex

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've raised some excellent points, @ziggie1984. I agree that the complexities introduced by maintaining compatibility across different SQL backends (Postgres and SQLite), coupled with the varied optional data for each hop (MPP, AMP, blinded), make a generic batch insert approach significantly more challenging than the potential performance gains for typical payment routes (2-10 hops).

The fact that the operations are already wrapped in a transaction mitigates some of the overhead, and for small numbers of hops, the current single-insert-per-hop within a transaction is indeed a reasonable and maintainable solution.

My initial suggestion was based on a general optimization principle, but your detailed explanation of the specific constraints and typical usage patterns demonstrates why a more complex batching mechanism might not be worth the effort here. I'm satisfied with the current implementation given these considerations. We can always revisit this if profiling indicates a bottleneck with significantly larger routes in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: can we build queries like INSERT INTO ... VALUES (...), (...), ...? I think it's doable for both postgres and sqlite? also need to know this for btcwallet work. cc @gemini-code-assist @ziggie1984

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, @yyforyongyu, you're correct. Both PostgreSQL and SQLite support multi-row INSERT statements using the INSERT INTO ... VALUES (...), (...), ... syntax. This can significantly improve insertion performance, especially when dealing with a large number of records. However, it's important to consider the limitations and potential complexities:

  • Statement Size Limits: Both databases might have limits on the size of a single SQL statement. Exceeding these limits can lead to errors. It's best to benchmark and determine a reasonable batch size.
  • Error Handling: With multi-row inserts, if one row fails to insert (due to a constraint violation, for example), the entire statement might be rolled back, depending on the database and transaction settings. You'll need to handle such errors appropriately.
  • Parameter Binding: When using prepared statements (recommended for security and performance), ensure that your database driver and query builder correctly handle parameter binding for multi-row inserts.

For the btcwallet work, this approach could be beneficial for inserting multiple addresses or transactions in a single operation. Just be mindful of the points above and test thoroughly.

@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 99ddf07 to 6145682 Compare October 17, 2025 08:43
@ziggie1984 ziggie1984 moved this from In progress to In review in v0.21 Oct 18, 2025
Comment on lines 540 to 776
completePayment, err := s.fetchPaymentWithCompleteData(
ctx, db, fetchPayment,
)
if err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think we need to fetch the complete payment here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah looks like overkill to get all the data, I think we only need completePayment.Status?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately we have to because it depends on the state of the HTLCs, so we need to always fetch the whole payment

se also func (m *MPPayment) setState() error {

Copy link
Member

@yyforyongyu yyforyongyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool and would appreciate if the CIs can pass first before requesting reviews, especially the linter.

Comment on lines 540 to 776
completePayment, err := s.fetchPaymentWithCompleteData(
ctx, db, fetchPayment,
)
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah looks like overkill to get all the data, I think we only need completePayment.Status?

@lightninglabs-deploy
Copy link

@ziggie1984, remember to re-request review from reviewers when ready

@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 6145682 to 047e180 Compare November 10, 2025 17:56
@ziggie1984 ziggie1984 moved this from In review to In progress in v0.21 Nov 10, 2025
@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 047e180 to 5f2b14a Compare November 11, 2025 10:29
Copy link
Collaborator Author

@ziggie1984 ziggie1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed all the feedback, I think the batch inserting I will do in a followup PR.

Comment on lines 540 to 776
completePayment, err := s.fetchPaymentWithCompleteData(
ctx, db, fetchPayment,
)
if err != nil {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately we have to because it depends on the state of the HTLCs, so we need to always fetch the whole payment

se also func (m *MPPayment) setState() error {

Comment on lines 645 to 908
completePayment, err := s.fetchPaymentWithCompleteData(
ctx, db, existingPayment,
)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above we need to make sure no HTLCs are INFLIGHT and none exist or all are failed basically.

@ziggie1984
Copy link
Collaborator Author

The itest for native_sql backends will not pass in this PR as well, they will start to pass from the next PR one (Part 4)

@ziggie1984 ziggie1984 force-pushed the elle-payment-sql-series-new branch from 9300d69 to 0288df7 Compare November 12, 2025 13:09
Copy link
Collaborator

@ellemouton ellemouton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

think maybe re-request once the diff is only the commits for this pr 🙏

thanks for the updates, should be g2g

-- When the payment is deleted, the intent is automatically deleted
payment_id BIGINT NOT NULL REFERENCES payments (id) ON DELETE CASCADE,

-- The type of intent (e.g. 0 = bolt11_invoice, 1 = bolt12_offer)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably name it bolt12_invoice? An offer is published by the receiver, and can generate many invoice_requests.


-- name: DeleteFailedAttempts :exec
DELETE FROM payment_htlc_attempts WHERE payment_id = $1 AND attempt_index IN (
SELECT attempt_index FROM payment_htlc_attempt_resolutions WHERE resolution_type = 2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could we add some docs explaining what is resolution_type 2?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

//
// TODO(ziggie): Refactor to not mix application logic with database
// logic. This decision should be made in the application layer.
if s.keepFailedPaymentAttempts {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm think we should rely solely on failedHtlcsOnly instead? I don't think this flag keepFailedPaymentAttempts is meant to be used in DeletePayment, as it's created such that the users don't need to manually delete the failed attempts for each payment, an automated GC provided by lnd.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I see what's going here, so previously, we didn't have a dedicated DeleteFailedAttempts, it uses the DeletePayment method underneath,

lnd/payments/db/kv_store.go

Lines 293 to 303 in 0491764

func (p *KVStore) DeleteFailedAttempts(hash lntypes.Hash) error {
if !p.keepFailedPaymentAttempts {
const failedHtlcsOnly = true
err := p.DeletePayment(hash, failedHtlcsOnly)
if err != nil {
return err
}
}
return nil
}

Given that we now have a dedicated DeleteFailedAttempts, I think we can ignore p.keepFailedPaymentAttempts and rely on failedHtlcsOnly only.

Copy link
Collaborator Author

@ziggie1984 ziggie1984 Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand tho, don't we need to adhere to the config setting for the sql store as well ? Let's followup with a TODO in a separate PR were we move the keepFailedPaymentAttempts out of the db logic ?

// hash already exists and validates whether initialization is allowed based on
// the existing payment's status:
// - StatusInitiated: Returns ErrPaymentExists (payment already created,
// HTLCs may be in flight)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this is possible, that a payment has the StatusInititated, yet it has inflight HTLCs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right, corrected.

Comment on lines +739 to +1058
func (s *SQLStore) insertRouteHops(ctx context.Context, db SQLQueries,
hops []*route.Hop, attemptID uint64) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: can we build queries like INSERT INTO ... VALUES (...), (...), ...? I think it's doable for both postgres and sqlite? also need to know this for btcwallet work. cc @gemini-code-assist @ziggie1984


// Insert blinded route data if present. Every hop in the
// blinded path must have an encrypted data record.
if hop.EncryptedData != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can exit early via if hop.EncryptedData == nil

return fmt.Errorf("failed to fetch payment: %w", err)
}

mpPayment, err = s.fetchPaymentWithCompleteData(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same question about whether it's necessary to load the full data, like we can build a query using the relevant tables to derive the status.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so for the register attempt we need to fetch the payment state which depends on the amount forwarded, which we fetch from the last hop, this could be improved to actually only fetch the last hop and don't fetch all the additional data, so I think I keeep this in mind if the performance tests can be improved ok ?


// Add the attempt to the payment without fetching it from the
// DB again.
mpPayment.HTLCs = append(mpPayment.HTLCs, HTLCAttempt{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if two concurrent goroutines are registering HTLC attempts at the same time? I think the db will have the correct result eventually, but one of the goroutines will have the wrong data? cc @gemini-code-assist

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great question! You're right to consider the concurrency implications here, @yyforyongyu.

In the current implementation, the RegisterAttempt function does use a database transaction to ensure atomicity. This means that even if two concurrent goroutines try to register an HTLC attempt for the same payment, the database will ensure that only one of them succeeds in writing the data, preventing race conditions at the database level.

However, your concern about one of the goroutines having the wrong data is valid. Although the database transaction ensures that the data is eventually consistent, one of the goroutines might read stale data before the transaction is committed. This could lead to issues if the goroutine relies on the updated state of the payment after registering the attempt.

To address this, you could consider adding a retry mechanism to the RegisterAttempt function. If the function fails to register the attempt due to a conflict (e.g., a database constraint violation), it could retry the operation after a short delay. This would give the other goroutine time to commit its transaction and update the database.

Another approach would be to use a locking mechanism to serialize access to the RegisterAttempt function for a given payment hash. This would ensure that only one goroutine can register an attempt for a payment at a time, preventing race conditions altogether.

Which approach is best depends on the specific requirements of your application. If you need to minimize latency, a retry mechanism might be preferable. If you need to ensure strict consistency, a locking mechanism might be a better choice.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just reload the payment in the end instead of trying this append from memory approach? @gemini-code-assist

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think reloading the payment at the end of the RegisterAttempt function is a reasonable approach, @yyforyongyu. It would ensure data consistency across concurrent goroutines, though it might add a bit of latency. It's a good trade-off to consider.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gemini-code-assist RegisterAttempt calls for the same payment hash must be serialized. This is typically done in the payment lifecycle manager (the router/payment controller), not in the database layer this cannot be enforced on the db layer otherwise we risk overpayment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right, @ziggie1984. Thank you for clarifying the architectural boundary and the critical importance of serializing RegisterAttempt calls at the payment router/controller layer to prevent overpayment. The added comment in payments/db/interface.go perfectly captures this crucial design decision and its implications. My previous suggestions were indeed focused on database-level solutions, but your explanation highlights why that's not the appropriate place for this particular concurrency control.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also added a not to the interface function.

return fmt.Errorf("failed to settle attempt: %w", err)
}

mpPayment, err = s.fetchPaymentWithCompleteData(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like these are two heavy reads, hmm...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

brought it down to 1 heavy read, the other one only fetches the status, here we might think if we want to always report the whole payment to the subscriber, but I think we can revisit this when we have performance data.

@ziggie1984 ziggie1984 force-pushed the elle-payment-sql-series-new branch 2 times, most recently from e991cd0 to 6860702 Compare November 13, 2025 06:45
@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 67e9d0d to 1b655be Compare November 13, 2025 06:47
@ziggie1984 ziggie1984 moved this from In review to In progress in v0.21 Nov 13, 2025
Previously a one(intent)-to-many(payment) relationship it is now
changed to a one-to-one relationship because a payment request
only can have 1 payment related to it. Looking into the future
with BOLT12 offers, the fetched invoice from the offer could be
stored here as well and the relationship would still hold.
@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 1b655be to 25413e3 Compare November 13, 2025 10:15
Copy link
Collaborator Author

@ziggie1984 ziggie1984 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As recommended by yy, I did improve on the payment status fetching for the sql case. The RegisterAttempt call however needs the last hop of the Route so I did not improve there for now, but I added a TODO to potentially revisit if the performance is not as great.

return fmt.Errorf("failed to fetch payment: %w", err)
}

mpPayment, err = s.fetchPaymentWithCompleteData(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so for the register attempt we need to fetch the payment state which depends on the amount forwarded, which we fetch from the last hop, this could be improved to actually only fetch the last hop and don't fetch all the additional data, so I think I keeep this in mind if the performance tests can be improved ok ?

return fmt.Errorf("failed to settle attempt: %w", err)
}

mpPayment, err = s.fetchPaymentWithCompleteData(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

brought it down to 1 heavy read, the other one only fetches the status, here we might think if we want to always report the whole payment to the subscriber, but I think we can revisit this when we have performance data.


// Add the attempt to the payment without fetching it from the
// DB again.
mpPayment.HTLCs = append(mpPayment.HTLCs, HTLCAttempt{
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gemini-code-assist RegisterAttempt calls for the same payment hash must be serialized. This is typically done in the payment lifecycle manager (the router/payment controller), not in the database layer this cannot be enforced on the db layer otherwise we risk overpayment.

@ziggie1984
Copy link
Collaborator Author

/gemini review

@ziggie1984 ziggie1984 moved this from In progress to In review in v0.21 Nov 13, 2025
Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request makes significant progress on implementing the SQL backend for the payments database by adding most of the required insert and delete methods. The database schema changes, particularly refactoring the payment_intents table, are a great improvement for data integrity and consistency. The new function comments are detailed and align well with the project's style guide.

My review focuses on a few areas for improvement:

  • Consistently using context.Context for request lifecycle management instead of context.TODO().
  • A couple of opportunities to optimize database queries for better performance by avoiding redundant or full data fetches when only partial data is needed.

Overall, this is a solid contribution to the SQL backend migration. Addressing these points will further improve the robustness and efficiency of the new implementation.

Comment on lines +755 to +767
func (s *SQLStore) DeleteFailedAttempts(paymentHash lntypes.Hash) error {
// In case we are configured to keep failed payment attempts, we exit
// early.
//
// TODO(ziggie): Refactor to not mix application logic with database
// logic. This decision should be made in the application layer.
if s.keepFailedPaymentAttempts {
return nil
}

ctx := context.TODO()

err := s.db.ExecTx(ctx, sqldb.WriteTxOpt(), func(db SQLQueries) error {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This function, and other new public methods on SQLStore (DeletePayment, InitPayment, RegisterAttempt, SettleAttempt), should accept a context.Context from the caller instead of using context.TODO(). This is crucial for request lifecycle management, including cancellation and deadlines. The PaymentControl interface should be updated to reflect this change.

func (s *SQLStore) DeleteFailedAttempts(ctx context.Context, paymentHash lntypes.Hash) error {
	// In case we are configured to keep failed payment attempts, we exit
	// early.
	//
	// TODO(ziggie): Refactor to not mix application logic with database
	// logic. This decision should be made in the application layer.
	if s.keepFailedPaymentAttempts {
		return nil
	}

	err := s.db.ExecTx(ctx, sqldb.WriteTxOpt(), func(db SQLQueries) error {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will be done in followup PRs.

In this commit we add all queries which we will need to insert
payment related data into the db.
@ziggie1984 ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 25413e3 to 8d382f4 Compare November 13, 2025 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

payments Related to invoices/payments sql

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

4 participants