Skip to content

Commit 106d0eb

Browse files
Rate-limit error logging for buffer overflow in input preprocessing.
Changes `LOG(ERROR)` to `LOG_EVERY_N(ERROR, 100)` to reduce log spam when the static buffer size is insufficient, as this condition can occur frequently. PiperOrigin-RevId: 817380757
1 parent 5dec751 commit 106d0eb

File tree

1 file changed

+11
-10
lines changed

1 file changed

+11
-10
lines changed

jax_tpu_embedding/sparsecore/lib/core/input_preprocessing_util.cc

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -69,16 +69,17 @@ struct BufferFillingOptions {
6969
bool ValidIndices(int row_index, int coo_offset, int processed,
7070
const BufferFillingOptions& options) {
7171
if (row_index >= options.lhs_row_end || coo_offset >= options.coo_end) {
72-
LOG(ERROR) << "The static buffer size might be too small for the current "
73-
"batch. IDs may be dropped! "
74-
<< "Stopping row pointer filling for local SparseCore ID "
75-
<< options.local_sc_id << " at row index: " << row_index
76-
<< " and coo offset: " << coo_offset
77-
<< " because it reached the end of the buffer. "
78-
<< "The buffer size is: " << options.coo_buffer_size
79-
<< ". The coo_end is: " << options.coo_end
80-
<< ". Total number of input COOs: " << options.coo_tensors.size()
81-
<< ", while currently processed only: " << processed;
72+
LOG_EVERY_N(ERROR, 100)
73+
<< "The static buffer size might be too small for the current "
74+
"batch. IDs may be dropped! "
75+
<< "Stopping row pointer filling for local SparseCore ID "
76+
<< options.local_sc_id << " at row index: " << row_index
77+
<< " and coo offset: " << coo_offset
78+
<< " because it reached the end of the buffer. "
79+
<< "The buffer size is: " << options.coo_buffer_size
80+
<< ". The coo_end is: " << options.coo_end
81+
<< ". Total number of input COOs: " << options.coo_tensors.size()
82+
<< ", while currently processed only: " << processed;
8283
return false;
8384
}
8485
return true;

0 commit comments

Comments
 (0)