Harvard University
|Data Engineer Intern
Highlights
Implemented web scraping algorithms like Automated and API based for IEEE and ACM conference papers
Scraped over 45,000+ research papers and calculated LLM tokens (200 million+) using OpenAI GPT-4 tokenizer.
The collected material is crucial to validate the model's accuracy using OpenAI Models(GPT-3.5, GPT-40, GPT-4)