-
Notifications
You must be signed in to change notification settings - Fork 729
[GH-2337] Fix the DateTimeParseException in GeoPackage reader #2339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
a636224
to
77ca076
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes a DateTimeParseException in the GeoPackage reader when processing datetime fields that lack timezone information. The fix enhances the datetime parsing logic to handle multiple formats and treat timezone-less strings as UTC.
- Enhanced the
epoch()
method with fallback parsing for datetime strings without timezone - Added support for various millisecond formats (1, 2, 3 digits, or none)
- Added comprehensive test coverage for the datetime parsing fix
Reviewed Changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
DataTypesTransformations.scala | Enhanced epoch() method with fallback parsing and support for multiple datetime formats |
GeoPackageReaderTest.scala (3 versions) | Added test case to verify datetime parsing without timezone information |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
for (formatter <- datetimeFormatters) { | ||
try { | ||
val localDateTime = LocalDateTime.parse(timestampStr, formatter) | ||
return localDateTime.toInstant(ZoneOffset.UTC).toEpochMilli | ||
} catch { | ||
case _: DateTimeParseException => | ||
// Continue to next formatter | ||
} | ||
} | ||
|
||
// If all formatters failed, throw a descriptive exception | ||
throw new IllegalArgumentException( | ||
s"Unable to parse datetime: $timestampStr. " + | ||
s"Expected formats: 'yyyy-MM-ddTHH:mm:ss[.S]' or 'yyyy-MM-ddTHH:mm:ss[.S]Z'") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] Using early return in a for loop is not optimal for functional programming. Consider using datetimeFormatters.view.map(...)
with find
or collectFirst
to make the code more functional and potentially more efficient.
for (formatter <- datetimeFormatters) { | |
try { | |
val localDateTime = LocalDateTime.parse(timestampStr, formatter) | |
return localDateTime.toInstant(ZoneOffset.UTC).toEpochMilli | |
} catch { | |
case _: DateTimeParseException => | |
// Continue to next formatter | |
} | |
} | |
// If all formatters failed, throw a descriptive exception | |
throw new IllegalArgumentException( | |
s"Unable to parse datetime: $timestampStr. " + | |
s"Expected formats: 'yyyy-MM-ddTHH:mm:ss[.S]' or 'yyyy-MM-ddTHH:mm:ss[.S]Z'") | |
datetimeFormatters.view | |
.map { formatter => | |
try { | |
Some(LocalDateTime.parse(timestampStr, formatter).toInstant(ZoneOffset.UTC).toEpochMilli) | |
} catch { | |
case _: DateTimeParseException => None | |
} | |
} | |
.collectFirst { case Some(epochMilli) => epochMilli } | |
.getOrElse { | |
// If all formatters failed, throw a descriptive exception | |
throw new IllegalArgumentException( | |
s"Unable to parse datetime: $timestampStr. " + | |
s"Expected formats: 'yyyy-MM-ddTHH:mm:ss[.S]' or 'yyyy-MM-ddTHH:mm:ss[.S]Z'") | |
} |
Copilot uses AI. Check for mistakes.
// If all formatters failed, throw a descriptive exception | ||
throw new IllegalArgumentException( | ||
s"Unable to parse datetime: $timestampStr. " + | ||
s"Expected formats: 'yyyy-MM-ddTHH:mm:ss[.S]' or 'yyyy-MM-ddTHH:mm:ss[.S]Z'") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error message format examples don't accurately reflect all supported patterns. It shows '[.S]' but the code supports .S, .SS, and .SSS formats. Consider updating to: 'yyyy-MM-ddTHH:mm:ss[.SSS|.SS|.S]' or 'yyyy-MM-ddTHH:mm:ss[.SSS|.SS|.S]Z'
s"Expected formats: 'yyyy-MM-ddTHH:mm:ss[.S]' or 'yyyy-MM-ddTHH:mm:ss[.S]Z'") | |
s"Expected formats: 'yyyy-MM-ddTHH:mm:ss[.SSS|.SS|.S]' or 'yyyy-MM-ddTHH:mm:ss[.SSS|.SS|.S]Z'") |
Copilot uses AI. Check for mistakes.
Did you read the Contributor Guide?
Is this PR related to a ticket?
[GH-2337] my subject
. Closes Error Reading geopackage (DateTimeParseException) and Showing metadata #2337What changes were proposed in this PR?
Problem: DateTimeParseException: Text '2023-12-31T12:00:00.000' could not be parsed at index 23
Root Cause: Instant.parse() requires timezone information, but GeoPackage datetime fields often don't include it
Solution: Enhanced the epoch() method with fallback parsing that treats datetime strings without timezone as UTC
Modified:
How was this patch tested?
Added 1 more test case and a hand-made geopackage file that contains problematic datetime data
Did this PR include necessary documentation updates?