Skip to content

Conversation

Samoed
Copy link
Member

@Samoed Samoed commented Sep 9, 2025

Fix #1877
Preparing for #2891

I've changed repositories of these datasets mostly.

Some exceptions in test_disallow_trust_remote_code_in_new_datasetswere previously reuploaded/changed or repositories don't require trust_remote_code

  • BornholmBitextMining
  • BibleNLPBitextMining
  • DiaBlaBitextMining
  • FloresBitextMining
  • IN22ConvBitextMining
  • NTREXBitextMining
  • IN22GenBitextMining
  • IndicGenBenchFloresBitextMining
  • IWSLT2017BitextMining
  • SRNCorpusBitextMining
  • VieMedEVBitextMining
  • HotelReviewSentimentClassification
  • TweetEmotionClassification
  • DanishPoliticalCommentsClassification
  • TenKGnadClassification
  • ArxivClassification
  • FinancialPhrasebankClassification
  • FrenkEnClassification
  • PatentClassification
  • PoemSentimentClassification
  • TweetTopicSingleClassification
  • YahooAnswersTopicsClassification
  • FilipinoHateSpeechClassification
  • HebrewSentimentAnalysis
  • HindiDiscourseClassification
  • FrenkHrClassification
  • Itacola
  • JavaneseIMDBClassification
  • WRIMEClassification
  • KorHateClassification
  • KorSarcasmClassification
  • AfriSentiClassification
  • AmazonCounterfactualClassification
  • AmazonReviewsClassification
  • MTOPDomainClassification
  • MTOPIntentClassification
  • NaijaSenti
  • NordicLangClassification
  • NusaX-senti
  • SwissJudgementClassification
  • MyanmarNews
  • DutchBookReviewSentimentClassification
  • NorwegianParliamentClassification
  • PAC
  • HateSpeechPortugueseClassification
  • Moroco
  • RomanianReviewsSentiment
  • RomanianSentimentClassification
  • GeoreviewClassification
  • FrenkSlClassification
  • DalajClassification
  • SwedishSentimentClassification
  • WisesightSentimentClassification
  • UrduRomanSentimentClassification
  • VieStudentFeedbackClassification
  • IndicReviewsClusteringP2P
  • MasakhaNEWSClusteringP2P
  • MasakhaNEWSClusteringS2S
  • MLSUMClusteringP2P.v2
  • CodeSearchNetRetrieval
  • DanFEVER
  • GerDaLIR
  • GermanDPR
  • AlphaNLI
  • ARCChallenge
  • FaithDial
  • HagridRetrieval
  • HellaSwag
  • PIQA
  • Quail
  • RARbCode
  • RARbMath
  • SIQA
  • SpartQA
  • TempReasonL1
  • TempReasonL2Context
  • TempReasonL2Fact
  • TempReasonL2Pure
  • TempReasonL3Context
  • TempReasonL3Fact
  • TempReasonL3Pure
  • TopiOCQA
  • WinoGrande
  • AlloprofRetrieval
  • BSARDRetrieval
  • BSARDRetrieval.v2
  • JaGovFaqsRetrieval
  • JaQuADRetrieval
  • NLPJournalAbsIntroRetrieval
  • NLPJournalTitleAbsRetrieval
  • NLPJournalTitleIntroRetrieval
  • IndicQARetrieval
  • MintakaRetrieval
  • MIRACLRetrieval
  • MLQARetrieval
  • MultiLongDocRetrieval
  • NeuCLIR2022Retrieval
  • NeuCLIR2023Retrieval
  • XMarket
  • XPQARetrieval
  • ArguAna-PL
  • DBPedia-PL
  • FiQA-PL
  • HotpotQA-PL
  • MSMARCO-PL
  • NFCorpus-PL
  • NQ-PL
  • Quora-PL
  • SCIDOCS-PL
  • SciFact-PL
  • TRECCOVID-PL
  • SpanishPassageRetrievalS2P
  • SpanishPassageRetrievalS2S
  • SwednRetrieval
  • SweFaqRetrieval
  • KorHateSpeechMLClassification
  • BrazilianToxicTweetsClassification
  • CTKFactsNLI
  • LegalBenchPC
  • indonli
  • OpusparcusPC
  • PawsX
  • XStance
  • MIRACLReranking
  • FinParaSTS
  • JSICK
  • JSTS
  • RonSTS
  • STSES
  • AlloProfClusteringP2P.v2
  • AlloProfClusteringS2S.v2
  • LivedoorNewsClustering
  • MewsC16JaClustering
  • MLSUMClusteringS2S.v2
  • FGVCAircraft
  • DigikalamagClassification
  • JapaneseSentimentClassification
  • FGVCAircraftZeroShot
  • DigikalamagClustering
  • LivedoorNewsClustering.v2
  • ParsinluEntail
  • ParsinluQueryParaphPC
  • PawsXPairClassification
  • JaCWIRRetrieval
  • NLPJournalAbsArticleRetrieval.V2
  • NLPJournalAbsArticleRetrieval
  • NLPJournalAbsIntroRetrieval.V2
  • NLPJournalTitleAbsRetrieval.V2
  • NLPJournalTitleIntroRetrieval.V2
  • MIRACLRetrievalHardNegatives
  • MKQARetrieval
  • NeuCLIR2022RetrievalHardNegatives
  • NeuCLIR2023RetrievalHardNegatives
  • DBPedia-PLHardNegatives
  • HotpotQA-PLHardNegatives
  • MSMARCO-PLHardNegatives
  • NQ-PLHardNegatives
  • Quora-PLHardNegatives
  • InfoSeekIT2ITRetrieval
  • VOC2007
  • JaCWIRReranking
  • JQaRAReranking
  • XGlueWPRReranking
  • MAUDLegalBenchClassification
  • MultiLongDocRetrieval
  • CanadaTaxCourtOutcomesLegalBenchClassification
  • ContractNLIConfidentialityOfAgreementLegalBenchClassification
  • ContractNLIExplicitIdentificationLegalBenchClassification
  • ContractNLIInclusionOfVerballyConveyedInformationLegalBenchClassification
  • ContractNLILimitedUseLegalBenchClassification
  • ContractNLINoLicensingLegalBenchClassification
  • ContractNLINoticeOnCompelledDisclosureLegalBenchClassification
  • ContractNLIPermissibleAcquirementOfSimilarInformationLegalBenchClassification
  • ContractNLIPermissibleCopyLegalBenchClassification
  • ContractNLIPermissibleDevelopmentOfSimilarInformationLegalBenchClassification
  • ContractNLIPermissiblePostAgreementPossessionLegalBenchClassification
  • ContractNLIReturnOfConfidentialInformationLegalBenchClassification
  • ContractNLISharingWithEmployeesLegalBenchClassification
  • ContractNLISharingWithThirdPartiesLegalBenchClassification
  • ContractNLISurvivalOfObligationsLegalBenchClassification
  • CorporateLobbyingLegalBenchClassification
  • CUADAffiliateLicenseLicenseeLegalBenchClassification
  • CUADAffiliateLicenseLicensorLegalBenchClassification
  • CUADAntiAssignmentLegalBenchClassification
  • CUADAuditRightsLegalBenchClassification
  • CUADCapOnLiabilityLegalBenchClassification
  • CUADChangeOfControlLegalBenchClassification
  • CUADCompetitiveRestrictionExceptionLegalBenchClassification
  • CUADCovenantNotToSueLegalBenchClassification
  • CUADEffectiveDateLegalBenchClassification
  • CUADExclusivityLegalBenchClassification
  • CUADExpirationDateLegalBenchClassification
  • CUADGoverningLawLegalBenchClassification
  • CUADInsuranceLegalBenchClassification
  • CUADIPOwnershipAssignmentLegalBenchClassification
  • CUADIrrevocableOrPerpetualLicenseLegalBenchClassification
  • CUADJointIPOwnershipLegalBenchClassification
  • CUADLicenseGrantLegalBenchClassification
  • CUADLiquidatedDamagesLegalBenchClassification
  • CUADMinimumCommitmentLegalBenchClassification
  • CUADMostFavoredNationLegalBenchClassification
  • CUADNoSolicitOfCustomersLegalBenchClassification
  • CUADNoSolicitOfEmployeesLegalBenchClassification
  • CUADNonCompeteLegalBenchClassification
  • CUADNonDisparagementLegalBenchClassification
  • CUADNonTransferableLicenseLegalBenchClassification
  • CUADNoticePeriodToTerminateRenewalLegalBenchClassification
  • CUADPostTerminationServicesLegalBenchClassification
  • CUADPriceRestrictionsLegalBenchClassification
  • CUADRenewalTermLegalBenchClassification
  • CUADRevenueProfitSharingLegalBenchClassification
  • CUADRofrRofoRofnLegalBenchClassification
  • CUADSourceCodeEscrowLegalBenchClassification
  • CUADTerminationForConvenienceLegalBenchClassification
  • CUADThirdPartyBeneficiaryLegalBenchClassification
  • CUADUncappedLiabilityLegalBenchClassification
  • CUADUnlimitedAllYouCanEatLicenseLegalBenchClassification
  • CUADVolumeRestrictionLegalBenchClassification
  • CUADWarrantyDurationLegalBenchClassification
  • DefinitionClassificationLegalBenchClassification
  • Diversity1LegalBenchClassification
  • Diversity2LegalBenchClassification
  • Diversity3LegalBenchClassification
  • Diversity4LegalBenchClassification
  • Diversity5LegalBenchClassification
  • Diversity6LegalBenchClassification
  • FunctionOfDecisionSectionLegalBenchClassification
  • InsurancePolicyInterpretationLegalBenchClassification
  • InternationalCitizenshipQuestionsLegalBenchClassification
  • LearnedHandsBenefitsLegalBenchClassification
  • LearnedHandsBusinessLegalBenchClassification
  • LearnedHandsConsumerLegalBenchClassification
  • LearnedHandsCourtsLegalBenchClassification
  • LearnedHandsCrimeLegalBenchClassification
  • LearnedHandsDivorceLegalBenchClassification
  • LearnedHandsDomesticViolenceLegalBenchClassification
  • LearnedHandsEducationLegalBenchClassification
  • LearnedHandsEmploymentLegalBenchClassification
  • LearnedHandsEstatesLegalBenchClassification
  • LearnedHandsFamilyLegalBenchClassification
  • LearnedHandsHealthLegalBenchClassification
  • LearnedHandsHousingLegalBenchClassification
  • LearnedHandsImmigrationLegalBenchClassification
  • LearnedHandsTortsLegalBenchClassification
  • LearnedHandsTrafficLegalBenchClassification
  • NYSJudicialEthicsLegalBenchClassification
  • OPP115DataRetentionLegalBenchClassification
  • OPP115FirstPartyCollectionUseLegalBenchClassification
  • OPP115InternationalAndSpecificAudiencesLegalBenchClassification
  • OPP115PolicyChangeLegalBenchClassification
  • OPP115ThirdPartySharingCollectionLegalBenchClassification
  • OPP115UserAccessEditAndDeletionLegalBenchClassification
  • PersonalJurisdictionLegalBenchClassification
  • PROALegalBenchClassification
  • SCDBPAccountabilityLegalBenchClassification
  • SCDBPAuditsLegalBenchClassification
  • SCDBPCertificationLegalBenchClassification
  • SCDBPTrainingLegalBenchClassification
  • SCDBPVerificationLegalBenchClassification
  • SCDDAccountabilityLegalBenchClassification
  • SCDDAuditsLegalBenchClassification
  • SCDDCertificationLegalBenchClassification
  • SCDDTrainingLegalBenchClassification
  • SCDDVerificationLegalBenchClassification
  • TelemarketingSalesRuleLegalBenchClassification
  • TextualismToolDictionariesLegalBenchClassification
  • TextualismToolPlainLegalBenchClassification
  • UCCVCommonLawLegalBenchClassification
  • UnfairTOSLegalBenchClassification

@Samoed Samoed added the v2 Issues and PRs related to `v2` branch label Sep 9, 2025
@Samoed Samoed marked this pull request as draft September 9, 2025 19:01
@Samoed
Copy link
Member Author

Samoed commented Sep 10, 2025

I accidentally found that all LegalBenchClassification tasks are required trust_remote_code (because 1 of 100 task specifies trust_remote_code inside load_data) and this is 100+ tasks. I don't understand why I didn't find this when I was computing descriptive_stats. I have only one version that I downloaded this repo once and manually agreed with trust_remote_code and this spread across all tasks, because they're all from the same repo

@Samoed Samoed marked this pull request as ready for review September 10, 2025 21:12
@Samoed Samoed changed the title start removing trust remote code Reupload datasets with trust remote code Sep 10, 2025
@Samoed
Copy link
Member Author

Samoed commented Sep 10, 2025

I've tested some classification, retrieval and bitext tasks and scores are the same with main

Copy link
Contributor

@KennethEnevoldsen KennethEnevoldsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonderful PR - great to see 5k deleted lines without a loss

just have a minor comment below

@KennethEnevoldsen KennethEnevoldsen merged commit 45114a5 into v2.0.0 Sep 19, 2025
8 checks passed
@KennethEnevoldsen KennethEnevoldsen deleted the reupload_datasts_with_trust_remote branch September 19, 2025 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v2 Issues and PRs related to `v2` branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants