NeurIPS Dataset Compiler
24 November 2022
#Web_scraping #GPU Optimization #Javascript
This was a mini-project which was born out of my desire to get the latest and greatest data for my Machine learning trend analysis project. Over here I extracted all the NIPS research papers through webscraping.
Although it was a big challenge, I enjoyed learning Javascript for the project. I used Python (BeautifulSoup) and JS to extract all NeurIPS research papers and when I felt the code is slow, I parallelized it using Multiprocessing library to reduce running time from 4+ hrs to 20 mins. (yay!)
I have made the code pubicly available through my Github (link below) and am really glad that its helping people around the world.