Extract meaningful words from a collection of PDF documents and count their frequencies
-
Updated
Jun 23, 2024 - Python
Extract meaningful words from a collection of PDF documents and count their frequencies
Fix mojibake and other Unicode glitches — a faithful TypeScript port of python-ftfy.
TypeScript port of python-ftfy for fixing mojibake and other Unicode text problems.
Built from scratch as a local RAG system for querying research papers using FAISS, Sentence Transformers, Ollama, and citation-backed retrieval.
Basic text preprocessing operations shown in jupyter notebook. You can play with them and look what are they doing. For stemming and lemmatization there are different options, I showed only what I prefer to use. Repository contains the data to play with taken from kaggle (can also be found here on github), but for convenience I attach it here.
To associate your repository with the ftfy topic, visit your repo's landing page and select "manage topics."