A few weeks ago, a friend asked me a very important question: “What do you do as a data scientist?”. I had to pause for a second, deciding whether I should refer him to the Wikipedia page for “data science”, explain to him what I think other data scientists might do right now, or tell him about my own career trajectory..
My programming skill set and CV is strong, I can speak 3 languages, I have worked in both start-ups and larger corporations and I have plenty of experience dealing with customers. Still, I struggled to find an answer and even more, I was struggling to find job postings which related to my skill set.
Throughout my job search however, I realized that the answer to the question of what modern “data scientists” do is quite broad and may be changing so rapidly that any past definition might be outdated.
Data Science Definitions and Origins
Wikipedia defines a data scientist as someone who “is a professional who creates programming code and combines it with statistical knowledge to create insights from data”.
That definition certainly leaves a lot to be desired. I have never come across a modern job posting which searches for someone who can use statistical insights to create insights from data as the only requirement, although perhaps this would have been possible 20 years ago. When dealing with the question of what data scientists do nowadays, we have to pay attention to not only the historical broadness of the definition, but how that definition is changing over time.
As far as I can tell, data science started coming into focus in the late 2000s, with “data science” courses beginning to be offered in university curriculum in mid-2010s. Data science was certainly not offered as a career path when I graduated in 2011. At that point, “computer science” was the only course phrasing which existed.
When I think back to my computer science courses, it was essentially creating plots and for/if loops to process data, so the terms “data science” in place of “computer science” likely arose from the rapid increase in data storage and processing capability which is ever increasing in our digital world, and certainly made leaps and bounds in the 2000s.
What did a data scientist in those early days do? Personally, I remember writing Python scripts to perform ETL into an SQL database, and writing R scripts to plot figures on a website in the late 2000s, but I already remember hearing rumblings of Hadoop, NoSQL, and large data lakes and warehouses. It took several more years for me to learn of neural networks and their impending applications.
By the late 2010s, my career placed me in an organization which had me filtering, structuring, modeling, and applying advanced mathematical methods to large amounts of data on Hadoop. While this was by no means the most advanced thing that a data scientist could be doing at that time, I took the most pride in the fact that I was able to translate and find solutions for our customers. After all, it was our customers that had the data that needed to be “scienced”.
We realized that often times, the customers would just dump a huge dataset at our feet without really knowing even what they wanted to do with it. My job was to clean and make sense of that data, and understand the problem enough to give suggestions about how they could use the data.
Changes in Landscape and Nomenclature
I realized that what I was doing as little as 5 years ago which was called “data science”, might nowadays be called “business intelligence”. The job title “data analyst” has also arisen, which seems to also be a more business-contextual and less model-intensive version of a data scientist. It is often the customer or recruiter who defines these positions on job posting boards, so if I personally have a hard time understanding the overlapping differences, it is sure that the recruiter or person posting the job will have similar struggles.
It has occurred to me in my recent job search, that there might be fewer data science job postings available. Obviously not due to less data being available, but rather due to the fact that data scientists have become more dispersed into different niche areas and/or are being classified in a different way.
The requirements for the more niche data science jobs of today are a giant leap from the requirement of having a vague mathematical background and some knowledge of Python, as might have been the case even 10 years ago. Job postings now require much more advance knowledge of specific topics, some of which currently seem to be the most common are:
- Experience with LLMs
- Neural Network and Deep Learning Model Tuning
- Generative AI Experience
- Image Processing Theory and Modeling Experience
The vetting process is also becoming more strict. I received three different job offers throughout the mid- and late 2010s purely through the interview process. Such a situation would be unthinkable today.
In most recent job applications, I have gone through at least 2 levels of both live and timed programmatic testing. I am not mentioning these points to complain, but only to emphasize how much the data science application process has changed and the level of specialization which is required now.
It is a natural consequence of any rapidly changing and ever more saturated field that new niches will be found and stricter testing implementations will be put into place. Even the current areas of expertise which I mentioned above are likely to change in the next 6 months.
During my last job, I found myself wondering how long “programming” as a data scientist related and value-added skill could exist, given that ChatGPT could now theoretically write all of my code for me in less than a minute. Perhaps this is why data scientists as well as companies and recruiters have begun rebranding and renaming their roles.
Job Searching as a Data Scientist Today
I have come to the conclusion that all-encompassing data scientists of the late 2000s and 2010s no longer exist, or at the very least are less in demand in the roles that they historically have played. There are frameworks to process data and write code more quickly (which is why data engineers are highly in demand, another term which I sometimes see interchanged with “data scientist” in job postings).
Data scientists now either do something similar to what I was doing in previous jobs, but with less emphasis on programming and more emphasis on business applications. Such a job usually falls under the field of “business intelligence” or “data analytics” now (as well as some companies still classifying this as “data science”).
A secondary option is that data scientists have become more specialized, again with less focus on programming and more on model tuning, knowledge of neural networks, deep learning, and the input parameters governing the output of these models.
In my opinion, these are indeed the two areas in which humans are still needed: in highly-specific model tuning and deep knowledge of model training using large amounts of data, and in taking very vague customer demands or ideas about unstructured data and using transformation, modeling, and visualization to produce business insights for them. This is where I believe “data scientists” currently are still relevant.
What does this mean for the job search? It doesn’t mean that data scientists are obsolete by any means, but only that the field has somewhat been rebranded in job postings. The person that used to be the all-encompassing data scientist could now fit into some/all of the following job postings:
- “Business Analyst”,
- “Data Analyst”,
- “Data Engineer”,
- “Model Developer”,
- “Neural Network Expert”,
- “Python Developer”,
- “Power BI / Visualization Expert.”
Obviously there are still job postings for “data scientist” available, and the number of these postings may even universally be increasing; however, in my experience the customer typically desires the same specific skills for a data scientist which now often can also fall under other, more specific job posting titles.
Once I realized the extent to which data science had expanded to other job titles, what job titles I needed to search for, and exactly what my skills with reference to those job titles are, my job search became much easier.
Conclusion
I remain very confident in my skill set after 10 years of working in the field; however, my job search has not been without effort. Much of that effort was knowing my personal skills, analyzing for several months how both the job and job posting landscape has changed, and taking the steps to rebrand myself and adjust my job searches accordingly. Perhaps it can help future data science employers and employees both to understand how quickly the landscape is changing.
-
For employees: Know your skillset, but don’t be afraid to search or apply to jobs which seem irrelevant (within reason) or that you perhaps feel unqualified for. Job titles are changing rapidly, and I have found that companies will often at least consider your application if you can prove that the skills you have could translate to one of the required technology stacks.
-
For employers: Understand that your job posting titles themselves may attract or repel some employees in this rapidly changing landscape. With that in mind, it may be good to at least consider a broader range of applications and experiences. I found in past jobs that the best employees were not the ones with the most experience, but the ones who could translate their existing experience to a new task and learn new topics quickly while doing it. With that said, It is often helpful for a data scientist to know the exact required technology stacks and years of experience required (as opposed to the requirements which are more flexible), and also the general purpose of the job posting and what role you want the employee to fulfill.
I am not advocating that my personal experience and opinions in this text are correct for everyone; but rather to share some information that I gained during my job search which hopefully makes the landscape navigable for not only freelancers, but also recruiters and prospective employers.
Need help with finding the right talent? CodeControl can support you either with remote hiring or with finding top tech freelancers.