Nice post, Andres! I am curious about a couple things: if you used Indeed or LinkedIn, how did you clean/ filter for ghost and expired job postings; and what are your observations on any changes in time-to-fill and experience levels? I wonder what is a reasonable way to evaluate those
I used SerpAPi, which pulls from Google Job Search based on a specific criterion. I set up my script to call the API daily, which meant I collected all active job postings during the period, but also lots of duplicates. To remove duplicates, I did the following:
(1) Select only unique jobs based on job_id
(2) Remove duplicates based on job title, company name, and description
This ensured I ignored reposted job ads. The experience level information mostly came from what was available in the job title (e.g, Senior Data Scientist), and I didn't look into "time-to-fill". Hopefully that was useful :)
Nice post, Andres!
Thank you, I appreciate it!
Nice post, Andres! I am curious about a couple things: if you used Indeed or LinkedIn, how did you clean/ filter for ghost and expired job postings; and what are your observations on any changes in time-to-fill and experience levels? I wonder what is a reasonable way to evaluate those
Hi Mounika,
I used SerpAPi, which pulls from Google Job Search based on a specific criterion. I set up my script to call the API daily, which meant I collected all active job postings during the period, but also lots of duplicates. To remove duplicates, I did the following:
(1) Select only unique jobs based on job_id
(2) Remove duplicates based on job title, company name, and description
This ensured I ignored reposted job ads. The experience level information mostly came from what was available in the job title (e.g, Senior Data Scientist), and I didn't look into "time-to-fill". Hopefully that was useful :)
Great post Andres 👏🏼
Thank you very much. I put a lot of work into it.
Insightful. Thank You!!
Great, glad you found it useful!