Skills needed to build a cloud based enterprise system that is secure, scalable and performant. Also, there’s a whole real-time aspect to interacting with this type of solution and inferencing has to be well designed.
great thinking! will add this to my list, coming up with a syllabus/defined set of skills for the AI Engineer is the most requested followup and something I purposely left open for others to weigh in haha
I needed this! I am a software engineer and started at an AI startup about a month ago. I was initially incredibly overwhelmed with all the "Deep Learning" and "ML" courses, and what not, out there. It wasn't until i started digging deeper into "what are people actually doing with this stuff?", that i started to realize there is a whole other side to this, and thus found a whole slew of tools, and practice projects to help me learn.
I think a tutorial around best practices and their evolution regarding various architectures like RAG, Agents, etc. would be 🔥, and maybe some examples of complex architectures with conditional agent / model routing?
I am working on a little personal project to learn all this stuff; i am building a browser text based adventure RPG based on The Call of Cthulhu with React UI, node/express/supabase, Langchain.js, and (eventually)Llama 2 (i am using GPT-4 in dev because it's a little faster to build with due to more examples online), where the "Game Master" and all of the NPC's are driven by a RAG "Agents". Through building this I am finding myself hacking a lot of things together and having a hard time finding examples of best practices to follow.
I'll take a look - though to your point, I don't think "best practices" really exist for the more complex systems yet! We're all sort of throwing things at the wall and figuring out what works in real-time.
I like where you are going. But one AI engineer persona knows how to build applications and platforms with all the new AI stuff. Aka, they need to understand how to create architectures that are reusable and scalable.
I think they answer this pretty well in another top-level comment but from my perspective, this is very much a matter of reliability and observability. As an example, for the production use-case I'm considering, I'm thinking about how to handle things like the unreliability of OpenAI's API, how to monitor the output of prompts to make sure that their quality is something we're comfortable with, and how to make sure that for folks who use our LLM micro-service have an API that is easy enough to use.
Fair! The unreliability of these models absolutely needs to be taken into account when designing systems and architectures. GPT-4's functions are a good example of this, since they aren't even guaranteed to return correctly-typed arguments 100% of the time.
Just as we've seen specialized roles emerge in the past two decades in the tech industry, we're bound to see a proliferation of specialized roles in the AI space.
This will be a combination of technical, ethical, regulatory, and product-focused roles, reflecting the multifaceted nature and impact of AI on society and industry.
Eng:
AI Engineer / ML Engineer --> While DS roles focus on building models, there's a growing realization that engineering robust AI systems is its own beast, leading to the rise of the AI or ML engineer role. These engineers focus on commercializing models, ensuring scalability, and integrating AI into broader software systems.
ML Research Engineer: This is a bridge between pure research scientists and production-focused software engineers. They transform experimental models from research papers into prototypes that can be tested in real-world scenarios.
MLOps Engineer: Analogous to DevOps in software engineering, MLOps focuses on the end-to-end ML lifecycle, ensuring smooth deployment, monitoring, and updating of ML models.
Infra:
New tools and platforms are constantly emerging to support the AI development lifecycle. Familiarity and expertise in using these tools, whether it's TensorFlow, PyTorch, MLflow, TFX, or newer platforms, are becoming vital.
100% ML Infra specialists are becoming a thing.
Governance:
As AI's impact on society becomes more pronounced, there's a growing need for specialists who understand the ethical implications of AI models and can design systems that are fair and transparent. These roles can be a blend of technical and philosophical expertise.
AI Product Managers:
Traditional product management is evolving to cater to AI-driven products. AI PMs need to understand the unique challenges of AI projects, from data acquisition to model interpretability, while aligning with business goals.
AI System Architects:
Just like system architects design complex software systems, AI architects will focus on designing intricate AI ecosystems, considering aspects like data pipelines, training infrastructures, inference optimization, and more.
Edge AI:
With AI moving to edge devices (like smartphones, IoT devices, etc.), there's a need for experts who can optimize models to run with limited resources without compromising performance.
AI Regulation and Policy Experts:
So many new classes and subsets of technical and non-technical roles are going to emerge from AI.
Skills needed to build a cloud based enterprise system that is secure, scalable and performant. Also, there’s a whole real-time aspect to interacting with this type of solution and inferencing has to be well designed.
great thinking! will add this to my list, coming up with a syllabus/defined set of skills for the AI Engineer is the most requested followup and something I purposely left open for others to weigh in haha
I needed this! I am a software engineer and started at an AI startup about a month ago. I was initially incredibly overwhelmed with all the "Deep Learning" and "ML" courses, and what not, out there. It wasn't until i started digging deeper into "what are people actually doing with this stuff?", that i started to realize there is a whole other side to this, and thus found a whole slew of tools, and practice projects to help me learn.
Thanks Treighton! If there are any additional tutorials or projects you're interested in seeing, let me know!
I think a tutorial around best practices and their evolution regarding various architectures like RAG, Agents, etc. would be 🔥, and maybe some examples of complex architectures with conditional agent / model routing?
I am working on a little personal project to learn all this stuff; i am building a browser text based adventure RPG based on The Call of Cthulhu with React UI, node/express/supabase, Langchain.js, and (eventually)Llama 2 (i am using GPT-4 in dev because it's a little faster to build with due to more examples online), where the "Game Master" and all of the NPC's are driven by a RAG "Agents". Through building this I am finding myself hacking a lot of things together and having a hard time finding examples of best practices to follow.
I'll take a look - though to your point, I don't think "best practices" really exist for the more complex systems yet! We're all sort of throwing things at the wall and figuring out what works in real-time.
Yeah, since I myself am new to all this I had no idea how new all this stuff is 😅
FWIW If there’s ever anyway i can help you out or collaborate on something I’m down!
I like where you are going. But one AI engineer persona knows how to build applications and platforms with all the new AI stuff. Aka, they need to understand how to create architectures that are reusable and scalable.
Can you say more? What kinds of AI platforms are you thinking about?
I think they answer this pretty well in another top-level comment but from my perspective, this is very much a matter of reliability and observability. As an example, for the production use-case I'm considering, I'm thinking about how to handle things like the unreliability of OpenAI's API, how to monitor the output of prompts to make sure that their quality is something we're comfortable with, and how to make sure that for folks who use our LLM micro-service have an API that is easy enough to use.
Fair! The unreliability of these models absolutely needs to be taken into account when designing systems and architectures. GPT-4's functions are a good example of this, since they aren't even guaranteed to return correctly-typed arguments 100% of the time.
Just as we've seen specialized roles emerge in the past two decades in the tech industry, we're bound to see a proliferation of specialized roles in the AI space.
This will be a combination of technical, ethical, regulatory, and product-focused roles, reflecting the multifaceted nature and impact of AI on society and industry.
Eng:
AI Engineer / ML Engineer --> While DS roles focus on building models, there's a growing realization that engineering robust AI systems is its own beast, leading to the rise of the AI or ML engineer role. These engineers focus on commercializing models, ensuring scalability, and integrating AI into broader software systems.
ML Research Engineer: This is a bridge between pure research scientists and production-focused software engineers. They transform experimental models from research papers into prototypes that can be tested in real-world scenarios.
MLOps Engineer: Analogous to DevOps in software engineering, MLOps focuses on the end-to-end ML lifecycle, ensuring smooth deployment, monitoring, and updating of ML models.
Infra:
New tools and platforms are constantly emerging to support the AI development lifecycle. Familiarity and expertise in using these tools, whether it's TensorFlow, PyTorch, MLflow, TFX, or newer platforms, are becoming vital.
100% ML Infra specialists are becoming a thing.
Governance:
As AI's impact on society becomes more pronounced, there's a growing need for specialists who understand the ethical implications of AI models and can design systems that are fair and transparent. These roles can be a blend of technical and philosophical expertise.
AI Product Managers:
Traditional product management is evolving to cater to AI-driven products. AI PMs need to understand the unique challenges of AI projects, from data acquisition to model interpretability, while aligning with business goals.
AI System Architects:
Just like system architects design complex software systems, AI architects will focus on designing intricate AI ecosystems, considering aspects like data pipelines, training infrastructures, inference optimization, and more.
Edge AI:
With AI moving to edge devices (like smartphones, IoT devices, etc.), there's a need for experts who can optimize models to run with limited resources without compromising performance.
AI Regulation and Policy Experts:
So many new classes and subsets of technical and non-technical roles are going to emerge from AI.
...and a whole lot else!