G.2073Page1GlobalResearchjournalofNaturalScience&Technology(GRJNST)Volume:04-Issue2(2026),2073ISSNP:2790-7643ISSNE:2790-7651www.grjnst.nethttps://doi.org/10.53762/grjnst.04.02.24TheHybridBERT-LSTMModelfortheclassificationSindhiTextinNLPReceived:27March2026.Accepted:3April2026.Published:27April2026NimraMemonLecturer,Dept.ComputerScience,Govt.GirlsDegreeCollegeNawabshahEmail:memonnimra03@gmail.comShabanaLecturer,Dept.ComputerScience,Govt.AishaGirlsDegreeCollegeNawabshahEmail:shabana.daudpoto@gmail.comWaqasAhmedMemonSoftwaredeveloper,atauxiliaryEmail:swt.waqas@gmail.comShahzadAyazMSEnglishlinguisticScholar,DepartmentofEnglish,QUESTNawabShahEmail:ayazshahzad03@gmail.comDuaaNoorMSCSscholar,Departmentofcomputerscience,DSU,KarachiEmail:duaanoorabbasi@gmail.comCorrespondingAuthor:DuaaNoor*(duaanoorabbasi@gmail.com)GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24Copyright©2026GRJNST.ThisarticleispublishedunderanOpenAccessmodel.ItismadeavailabletothepublicunderthetermsoftheCreativeCommonsAttribution4.0International(CCBY4.0)license,whichpermitsunrestricteduseanddistributionG.2073Page2Abstract:ThetraditionalMLmodelslacktocapturetherelationshiphavingdeepsemanticnature,whiledeeplearningmodelalonecannotworkbetterwithtemporalandcontextualembeddings.InthiscontexttheneedofefficientHybridapproachBERT-LSTMfortheimprovementofthetextclassification.ThisstudyproposestheHybridapproachBERT-LSTMonthesindhitextdata.Thetextdataiscollectedinsindhilanguagefromhuggingface.Thedatasetcontainsthelabeledsamplesofthesindhilanguagetexthavingtheirpredefinedclasses.Total150sentencesareusedforthesindhitextclassification.Themodelperformedrobustperformanceresultsbytheall-evaluationmatrices,whichachievedmacro-averageof0.88,0.88accuracyand0.86precisionandrecall0.85.thesignificantuseofthemacro-averagebecauseitconfirmstheconsistentmodelpredictiveabilityacrossthesentimentaldatatextualclasses.TheBERTembeddingsprovidesustainablegranularityinsindhitextsyntaxmightprovidethemissclassificationwithisshowninsenseofminimaldispersionoff-diagonalcells.ThisstudyprovidesthecriticalgapinsentimentalanalysisforthesindhitextdatabyprovidingthehybridapproachBERT-LSTMmodelarchitecture.ThemultilingualBERTisprovidedtoaddforfeatureextractionandforthemodelingcapabilityforthesequentialcapabilitytheBidirectionalBERTisused.ThesemanticnuanceandthelowofsindhitextstructuralbehavioriseffectivelycapturedbytheHybridapproach.Keywords:BERT,LSTM,NLP,sindhilanguageandSentimentanalysis1.1Introduction:TheadventofLargeLanguageModels(LLMs)hasmarkedrevolutionaryadvancementsinartificialintelligence,providingsystemswithemergentcapabilitiesinsophisticatednaturallanguageunderstandingandgenerationacrossamultitudeofscientificandcommercialdomains[1].Totransitionthesemodelsfrompowerfullinguisticstoolstoindispensableknowledgeworkers,however,theirfunctionalitymustextendbeyondsimplefactoidretrievaltowardcomplex,compositionalreasoning.Multi-hopequestionanswering(QA)representsGRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073Page3theapexofthischallenge.Successfulmulti-hopQArequirestheagenttoexecutearigoroussequenceofcognitivesteps,includingreadingcomprehension,logicalinference,andtheaccurateintegrationandsynthesisofdisparateknowledgeunits.LLMs,particularlythosereplyingonsequentialgenerationmethodslikeChain-of-Thought(CoT),oftenmimicthefast,intuitivedecision-makingobservedinhuman“System1”cognition.Thisinherentrelianceonsequential,token-leveldecision,evenwhenaugmentedbyrudimentaryplanning,rendersthemodelshighlysusceptibleaccumulatingerrors[2].Thestructuraldeficienciesmanifestasspecificfailuremodes,includingpoordynamicknowledgeadaptabilityandsignificantknowledgeintegrationerrors,suchasgeneratingincorrectrelationaljumpsduetovectorcombinationinaccuraciesduringintermediatereasoningsteps.Figure1ApplicationsofNLPCrucially,inthecontextofmulti-hoptasks,errorpropagationissignificantlyamplified.Aminordeviationorinaccuracyintroducedearlyinthereasoningchain,weatherduringinitialretrieval,contentinterpretation,orthefirstlogicaljump–compoundsexponentially,resultingincatastrophicfailuresinthefinalgeneratedoutput[3].ThisfragilitydemonstratesthatthelimitationsincurrentLLMmulti-hopperformancearenotsolelyattributabletoretrievalshortcomings[4].Empiricalanalysisrevealsthatachievingevenperfectretrievalaccuracy,whereallnecessarycontextsisprovided,doesnoteliminatereasoningerrors,emphasizingthatthecentralchallengeliesinthecompositionalstructureandlogicalverificationoftheinferenceprocessitself[5].Consequently,anyviablesolutionmustGRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073introducearchitecturalchangesthatenforceaverifiable,logicalflowtoensurestructuralsoundness,ratherthansimplyoptimizingthequantityorqualityofretrievedinformation.Page42.LiteratureReview:TheBERTmodelisusedfortheclassificationofthetextandfocusesgenerativeAIBasedapproachin[6].Thestudyin[7]Theretrieval-AugmentedGeneration(RAG)paradigmsuccessfullyaddressesthefundamentalissueofLLMhallucinationbygroundedgenerationinexternal,up-to-dateknowledgesources[8].TraditionalRAGsystemsoperatethroughastraightforward,fixedsequence:Thequeryisreceived,relevantdocumentsareretrieved,andthedocumentsaugmentthepromptforgeneration.Whileeffectiveforstraightforwardqueries,thisstatic,one-shotretrievalandgenerationmodelquicklyprovesinadequateforthedemandsofmulti-stepreasoning.Forcomplex,multi-hopqueries,theRAGpipelinebreaksdownduetoitsinabilitytoadaptmid-process.Traditionalsystemsarenotdesignedtohandlecomplexrequirementslikedynamiccontextualchanges,comparisonacrossmultipledatasets,oriterativerefinementofretrievalbasedonintermediateresults[6].Theyrelyonasingle,fixedretrievalpathandlackthenecessaryautonomousinterventiontoaddressretrievalissuesorcontextualdriftthatmayariseduringamulti-stepgenerationtask.TheempiricaldifficultyofthesetasksissubstantiatedbystandardizedbenchmarkssuchasHotpotQAand2WikiMultiHop,whichareexplicitlydesignedtotestthemodel’sabilitytosynthesizeevidenceacrossmultipledocuments[9].Forinstance,questionswithintheHotpotQAdatasetareconstructedsuchthattheirresolutionnecessitatesbridginginformationfoundinintroductoryparagraphsoftwoseparateWikipediaarticles,requiringthemodeltodemonstratetruerelationalandsyntheticreasoningacrosstextualboundaries[10].Thisrequirementfordeep,knowledge–extensivecompositionunderscoresthenecessityforarchitecturalinnovationsthatcanmanageandverifycomplexreasoningtrajectorieseffectively[11].AgenticRetrieval-AugmentedGeneration(AgenticRAG)representsthatnecessaryarchitecturalevaluationtoovercomethesestaticconstraints.AgenticRAGintegratesautonomousAIagentscapableofreasoning,goal-drivenbehavior,dynamicplanning,andGRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073Page5tooluseintotheknowledgeworkflow[12].Inthisadvancedstructure,TheRAGmechanismisre-contextualizednotasafixedpipelinestagebutasasophisticatedtooldynamicallymanagedbyaplanningagent.Thisdynamicorchestration,facilitategenuinemulti-stepproblem-solving[13].Agenticsystemsinherentlybreakdowncomplextasksintosmaller,executablesub-tasks,allowingthesystemtoadaptitsstrategyonthefly.Thisincludesdynamicallyselectingandqueryingmultiple,specializedknowledgesources,callingexternalAPIs,oriteratingretrievaleffortsbasedonongoingresults,achievingalevelofflexibilityandaccuracyfarexceedingtraditionalRAG[14].Theefficacyofthisdynamicapproachisdemonstratedinstate-of-the-artarchitectureslikerecursiveevaluationandadaptiveplanning(REAP)[15].REAPemploysadual-moduleframework.thesub-taskplanner(SP)andthefactextractor(FE)-linedbyanexplicit,recursivefeedbackloop.TheSPmaintainsaglobalperspective,activelyguidingtheoverallreasoningtrajectorytoavoidthelocalreasoningimpassescommoninmyopic,stepbystepsystems.TheSPevaluatesthetaskstatebasedonthefulfillmentleveloffactsextractedbytheFE,creatingamutuallyreinforcingcyclethatenablesplanningandreasoningcapabilities.However,theadoptionoftheAgenticRAGparadigmintroducesacorrespondingcomplexityandassociatedcomputationaloverhead[16].Thenecessityformulti-agentcollaboration,dynamictool-calling,andmultiplerecursiveLLMinteractionssignificantlyincreasesresourcerequirements,latency,andcoordinationcomplexitycomparedtotraditionalRAGpipelines[17].Thiselevatedcostmandatesthatanysubsequentoptimization,particularlyconcerningrecursiveerrormitigation,mustbehighlytargetedandefficient.Thesystemcannotaffordtoinvokehigh-latencyverificationchecksaftereveryintermediatestep;interventionmustbestrategicandpredictedonahigh-riskassessment.3.ProblemStatement:TheclassificationoftextinNLPrequiredthepowerfulmodelswhichcanexhibitsandperceivesthecontextualunderstandingwithmeaningfulandsequentialtextdependencies.ThetraditionalMLmodelslacktocapturetherelationshiphavingdeepsemanticnature,whiledeeplearningmodelalonecannotworkbetterwithtemporalandcontextualembeddings.InthiscontexttheneedofefficientHybridapproachBERT-LSTMfortheGRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073improvementofthetextclassificationresultsintermsofaccuracybyaddingBERTscontextuallanguagewithLSTMSequentialmodelingapproachability.Page64.Objective:ToproposeanddesignhybridapproachBERT-LSTMfortheclassificationofsindhilanguagetext.Thisextractionofsequentialandcontextualfeaturesfromthetextualdatatoimprovethetextclassificationonsentimentaldataanalysisinsindhilanguage.5.Methodology:a.Datacollection:Thetextdataiscollectedinsindhilanguagefromhuggingface.Thedataispreparedandorganizedinwell-structuredmannersinsingletextfile.Thedatasetcontainsthelabeledsamplesofthesindhilanguagetexthavingtheirpredefinedclasses.Total150sentencesareusedforthesindhitextclassification.ThedatasetviewisgivenbelowinfigureGRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24Figure2MethodologyandworkplanofthestudyG.2073Page7Figure3SindhiLanguagetextdatasetviewb.Datapreprocessing:Thedatapreprocessingisappliedtoremovesymbolsandnoisewhichisunnecessarilyaddedduringdatacollectionphase.Thetextisconvertedintoproperinputformatwhichisacceptableforthetokenization.TheBERTmodelisusedforthetexttokenization,withfurtheraddedthepaddingandtruncatingforthelengthfixing.Thedatasetisfurtherspiltedintotesting,training,andvalidationsets.c.Modeldesign:Thehybridmodelisdevelopedforthesindhilanguagetextclassification.TheBERTisusedforthetextgenerationandfortheembeddingstheinputtextwithadditionalLSTMmodelisfurtherusedforthesequentialinputfromtheBERTmodelforcapturingtemporalandsequentialdependencies.AdenselayerisaddedwiththeactivationfunctionnamedSoftMaxforthepredictionofthefinalclass.D.Modeltraining:Themodeltrainingisperformedwithadamoptimizerandthecategoricalcross-entropyisused.Thehypermeterstechniquesareusedforthebettermodeltrainingasbatchsize,epochnumber,learningrate,andunitshiddenareusedtotunedformodeloptimization.Forthereducingoverfittingdropoutorregularizationisused.GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073Page8d.Performanceevaluation:Performanceevaluationisprovidedonthebasisofdataset.threeclassesarenegative,neutralsentimentandpositive.Themodeltrainingandtestingisperformedonthebasisoftheevaluationmatrices.TheevaluationMatrixprecision,recall,accuracyandf1scorebelowinfigure3.ThegraphisgeneratedtoHybridmodelperformanceevaluationonthesindhitextdata.Figure4BERT+LSTMPerformancematrixTheHybridmodelapproachBERT-LSTMarchitectureisevaluatedwiththe150balanceddatasetofsindhilanguagesentences.Themodelperformedrobustperformanceresultsbytheall-evaluationmatrices,whichachievedmacro-averageof0.88,0.88accuracyand0.86precisionandrecall0.85.thesignificantuseofthemacro-averagebecauseitconfirmstheconsistentmodelpredictiveabilityacrossthesentimentaldatatextualclasses.Astheclassesarepositive,neutralandnegative),whereastheratherthanminorityclassesormajorityclasses.ThisshowstheHybridmodelapproachisprovidingthegoodresultsandhavingpotentialtrainingphasesbiases,duetotheLSTMbidirectionallayer,forthesequentialinputdatapatternswhichinfluencesonthedatapatterns.e.Confusionmatrix:GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073Theperformanceevaluationisachievedbythefindingconfusionmatrixalsointhisstudytocheckthecorrectpredictionclasses.Thefigure4belowshowsthenegative,predictiveandposwithactualvspredictedconfusionmatrixvalues.ThehybridModelBERT+LSTMshowstheresultsinconfusionmatrixgraphasgivebelowinfigure4.Page9Figure5ConfusionmatrixofHybridModelBERT-LSTMModelThefigure4furthershowstheinsightofthemodelsClassificationabilitywithaccuracyanditspattersoferrors.Thediagonalconcentrationvaluesensurethesentimentinstancesareclassifiedcorrectlywithmajorityofthesentiment.Themodel’sabilitytodistinguishtheclassesasneutralandnegativewithsentimentofthetext,isverycriticalchallengewithweakresourcesinNLP.TheBERTembeddingsprovidesustainablegranularityinsindhitextsyntaxmightprovidethemissclassificationwithisshowninsenseofminimaldispersionoff-diagonalcells.Themisclassificationofinstancesisprimarilyprohibited,wheresentence’sstructureisshortlength,whereassequentialcontextislimitedwithavailabilitytothelayerofLSTM.5.TheSynergisticefficiencyoftheHybridarchitecture:TheexperimentgainstheoptimizedperformancetothesynergisticapproachbetweentherelationshipsoftheBERTandLSTMcomponents:GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073a.b.Contextualextractionofthefeatures(BERT):thisisachievedbytheutilizingthebert-base-multingual-cased,themodelcapitalizedthelinguisticspretendknowledge,whichextractingthecontextualinformationandfeatureswhichawarethesindhilanguagemorphologicalnature.Page10SequentialpatternModeling(LSTM):ThetokenlevelrepresentationiscapturedbytheBERTandsequentialdependenciesarecapturedandmodeleffectivelybythebidirectionalLSTMmodellayers.Thesentiment“trajectory-capturingsentimentacrossthesentencesshiftaremodeltomaintain,whichprovidetheadvantageforthestandardlinearheadofclassification.c.overfittingmitigation:Thegivendatasizeof150sentences,whichprovidestheoverfittingrisk.TheregularizationtechniquesimplementationtermedasBERTprovidesthecriticallayerfreezingandearlystopping.TheBERTmodelpretrainedfreezingweights,thetrainableparametersarerestrictedtotheLSTMandheadfortheclassification.Thisensuresthestructuralnuancesofthemodelwhichisprioritizedthelearningprocesseffectivelytowardsthesentimentbearingsequences,havingwithoutalteringtherobust,andpretrainedsemanticunderstandingwhichismultilingualBERTbackingisprovidedtoworkwithit.6.Conclusion:ThisstudyprovidesthecriticalgapinsentimentalanalysisforthesindhitextdatabyprovidingthehybridapproachBERT-LSTMmodelarchitecture.ThemultilingualBERTisprovidedtoaddforfeatureextractionandforthemodelingcapabilityforthesequentialcapabilitytheBidirectionalBERTisused.ThesemanticnuanceandthelowofsindhitextstructuralbehavioriseffectivelycapturedbytheHybridapproach.GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073References:Page11[1]M.Alawida,S.Mejri,A.Mehmood,B.Chikhaoui,andO.IsaacAbiodun,“AComprehensiveStudyofChatGPT:Advancements,Limitations,andEthicalConsiderationsinNaturalLanguageProcessingandCybersecurity,”Inf.,vol.14,no.8,2023,doi:10.3390/info14080462.[2]T.Adão,A.Chojka,D.Pascoal,N.Silva,R.Morais,andE.Peres,“SyntheticData-DrivenMethodstoAcceleratetheDeploymentofDeepLearningModels:ACaseStudyonPestandDiseaseDetectioninPrecisionViticulture,”Computers,vol.14,no.8,pp.1–25,2025,doi:10.3390/computers14080327.[3]A.Katharria,K.Rajwar,M.Pant,J.D.Velásquez,V.Snášel,andK.Deep,“InformationFusioninSmartAgriculture:MachineLearningApplicationsandFutureResearchDirections,”2025,[Online].Available:http://arxiv.org/abs/2405.17465[4]A.Kovari,“AsystematicreviewofAI-poweredcollaborativelearninginhighereducation:Trendsandoutcomesfromthelastdecade,”Soc.Sci.Humanit.Open,vol.11,no.August2024,p.101335,2025,doi:10.1016/j.ssaho.2025.101335.[5]Y.Lei,J.Li,Z.Li,Y.Cao,andH.Shan,“Promptlearningincomputervision:asurvey,”Front.Inf.Technol.Electron.Eng.,vol.25,no.1,pp.42–63,2024,doi:10.1631/fitee.2300389.[6]E.Mikołajewska,D.Mikołajewski,T.Mikołajczyk,andT.Paczkowski,“GenerativeAIinAI-BasedDigitalTwinsforFaultDiagnosisforPredictiveMaintenanceinIndustry4.0/5.0,”Appl.Sci.,vol.15,no.6,pp.1–22,2025,doi:10.3390/app15063166.[7]S.Sandiwarno,D.I.Sensuse,H.B.Santoso,D.S.Hidayat,A.S.Nyamawe,andA.Yousif,“E-SATNet:EvaluatingStudentSatisfactionwithLecturerResponsesinAsynchronousOnlineDiscussionsUsingSentimentandSemanticSimilarityAnalysis,”BigDataCogn.Comput.,vol.9,no.9,pp.1–40,2025,doi:GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.207310.3390/bdcc9090228.Page12[8]K.W.Church,J.Sun,R.Yue,P.Vickers,W.Saba,andR.Chandrasekar,“Emergingtrends:AgentleintroductiontoRAG,”Nat.Lang.Eng.,vol.30,no.4,pp.870–881,2024,doi:10.1017/S1351324924000044.[9]S.Arora,V.Kumar,andT.Sabo,“DesigningSoftwareWithArtificial,”vol.08,no.03,pp.839–846,2021.[10]P.Weber,K.V.Carl,andO.Hinz,ApplicationsofExplainableArtificialIntelligenceinFinance—asystematicreviewofFinance,InformationSystems,andComputerScienceliterature,vol.74,no.2.SpringerInternationalPublishing,2024.doi:10.1007/s11301-023-00320-0.[11]I.Zafat,A.Iqbal,M.Khan,N.Ahmad,andM.AliAlshara,“GenIIoT:GenerativeModelsAidedProactiveFaultManagementinIndustrialInternetofThings,”Inf.,vol.16,no.12,pp.1–27,2025,doi:10.3390/info16121114.[12]W.Zita,S.AbouElFaouz,M.Alayedi,andE.E.Elsayed,“AHybridBayesianMachineLearningFrameworkforSimultaneousJobTitleClassificationandSalaryEstimation,”Symmetry(Basel).,vol.17,no.8,pp.1–24,2025,doi:10.3390/sym17081261.[13]A.C.Review,“Retrieval-AugmentedGeneration(RAG)inHealthcare:,”pp.1–29,2025.[14]S.M.D.A.C.JayatilakeandG.U.Ganegoda,“InvolvementofMachineLearningToolsinHealthcareDecisionMaking,”J.Healthc.Eng.,vol.2021,2021,doi:10.1155/2021/6679512.[15]F.Cuconasuetal.,“ThePowerofNoise:RedefiningRetrievalforRAGSystems,”SIGIR2024-Proc.47thInt.ACMSIGIRConf.Res.Dev.Inf.Retr.,no.July2024,pp.719–729,2024,doi:10.1145/3626772.3657834.[16]M.Arslan,H.Ghanem,S.Munawar,andC.Cruz,“ASurveyonRAGwithLLMs,”GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24G.2073ProcediaComput.Sci.,vol.246,no.C,pp.3781–3790,2024,doi:10.1016/j.procs.2024.09.178.[17]C.Krupitzer,“Generativeartificialintelligenceintheagri-foodvaluechain-overview,potential,andresearchchallenges,”Front.FoodSci.Technol.,vol.4,no.September,pp.1–6,2024,doi:10.3389/frfst.2024.1473357.Page13GRJNST,Volume:04-Issue2(2026)/ISSNP:2790-7643ArticleID:2073https://doi.org/10.53762/grjnst.04.02.24