I need to process Excel files received from multiple vendors and match the data against records stored in a database. The main challenge is that the Excel data is often inconsistent and unstructured. It contains variations such as mismatched words, abbreviations, spelling errors, and improper use of grammar or prepositions. Due to these inconsistencies, direct matching with database records becomes difficult. I need a solution that can intelligently identify and match the correct records despite these variations, ensuring high accuracy in mapping Excel data to the corresponding standardized database entries.
Hello @Sathwik Karamsetty
This seems a lot like a process problem. Usually the best path is to fix the process to guarantee the data consistence. Because any other solution will have (most likely) a permanent cost (and may not be cheaper).
If fixing the process is not possible, I think there is no magic bullet here. If your data is inconsistent, you probably need something like an AI Agent to try and sort out the data.
Out of the box this is not available in Traditional Web applications, only in ODC.
Depending on your license, you can request an ODC infrastructure and implement an Agent to consume in your Web Application (don't remember if you can expose an API on the agent or if you need an app for that).
Another possibility would be to connect your web app directly with a model, implementing the model API.
You can also try to find a service that provides this feature for you.
Any of these solutions will add cost to the application usage, of course.
And in any case, 100% accuracy is most likely not happen. You will need a human in the loop to guarantee everything is right or at least to take a look at anything flagged.
Don't have experience with this, so I can't point any other solutions.Maybe someone can come up with something different and "easier/not expensive".
Cheers
I´d say that if your comparision needs to consider "mismatched words, abbreviations, spelling errors, and improper use of grammar or prepositions" any AI you use will probably be non deterministic and may give more negative matches than what you expect.