Work In Tech

Find your next role at Canada's fastest-growing tech companies

Machine Learning Engineering Manager, LLM Ops

Klue

Klue

Software Engineering, Other Engineering, Operations
Toronto, ON, Canada
Posted on Oct 9, 2024

👋 Klue Engineering Leadership is hiring!

The Klue team is growing, and we're looking for an Engineering Manager to lead our LLM Ops / ML Ops team located in Amsterdam and Canada. In this role, you'll lead a team of Backend + Machine Learning engineers to support and build the future of Competitive Enablement.

💡 FAQ

Q: Klue who?

A: Klue is a VC-backed, capital-efficient growing SaaS company. Tiger Global and Salesforce Ventures led our US$62m Series B in the fall of 2021. We’re creating the category of competitive enablement: helping companies understand their market and outmaneuver their competition. We benefit from having an experienced leadership team working alongside several hundred risk-taking builders who elevate every day.

We’re one of Canada’s Most Admired Corporate Cultures by Waterstone HC, a Deloitte Technology Fast 50 & Fast 500 winner, and recipient of both the Startup of the Year and Tech Culture of the Year awards at the Technology Impact Awards.

Q: What is the focus of this team?

A: This team is responsible for enabling all teams at Klue to generate insights from large datasets using a combination of classic ML and Gen AI technologies. Current projects include LLM Ops tooling to enable teams to track and inspect their LLM usage, define and deploy evals for prompt optimization, and quickly iterate on zero/few shot prompts. We're also exploring model distillation, taking insight generators developed by Prompt and ML Engineers and creating smaller, faster, and more cost-effective models. Our ML Ops efforts include monitoring and retraining of our 5+ custom ML models currently deployed to production. Additionally, the team is building the infrastructure for generating insights at scale, enabling automated insights from diverse data sources such as news, competitor website changes, call transcripts, reviews, emails, PDFs, decks, and documents.

Q: What technologies do we use?

A: Python, Transformers, Pytorch, Hugging Face, LLMs, GPT, Bedrock, Spacy, Sklearn, Pinecone, Kubeflow, Vertex AI, Langfuse, BentoML, JS, PostgreSQL, Elasticsearch, Redis, GCP, BigQuery, Docker/Kubernetes, Github. We believe in using whatever tools make sense to get the job done and support our game-changing innovation.

Q: What are the responsibilities, and how will I spend my time?

- 30% People Leadership (recruiting, hiring, mentoring, 1:1s, career progression)

- 20% Project Management (backlog grooming, retrospectives, prioritization, milestone planning and management)

- 50% Technical Leadership (you'll work with others to shape Klue's engineering organization...architecture, quality control, tech debt, code review, etc) + Individual Contributions (you're not afraid to get your hands dirty and dig into our codebase...software engineering, testing, deployments, etc)

Q: What experience are we looking for?

  • We’re looking for someone with experience in technical and people leadership and with proven experience delivering high-value outcomes and strategy through solid engineering management.

  • 2+ years with direct management responsibilities with an engineering team

  • 5+ years of total software engineering experience, with a focus on Machine Learning

  • BS, MS, or PhD in Computer Science or a related technical field involving coding (e.g., physics or mathematics), or equivalent technical experience

  • Experience setting a vision and iteratively delivering progress to it

  • Ability to mentor team members on having the highest possible impact

  • Strong analytical capabilities to optimize solutions in a space with a lot of complex moving pieces, variables, and innovations

  • You have worked at a fast-growing start-up or SaaS with high-volume data and are eager to contribute in such an environment

Q: What aspects are we considering when evaluating level?

  • Impact: Ensuring your team delivers extraordinary output, and continuously seeks ways to make an outsized impact.

  • Innovation + Agility: Making the team nimble by encouraging bold and innovative approaches, growth mindset, experimentation, and listening to customer insights to achieve better results.

  • Decision Making: Strategic thinking, ability to break down complex problems into simple solutions, and leveraging insights and working cross-functionally with partners from our Product Team.

  • Team Development: Growing and developing an intellectually curious team of hardworking and motivated engineers with high expectations around individual ownership and impact.

How We Work at Klue:

  • Hybrid. Best of both worlds (remote & in-office)

  • Our main Canadian hubs are in Vancouver and Toronto. Ideally, this role would be located in Toronto.

  • You and your team will be in office at least 2 days per week.

Q: What about total compensation & benefits?

  • Benefits. We currently have extended health benefits starting on your 1st day.

  • Time off. Take what you need. We want the team to prioritize wellness and avoid burnout. Vacation usually falls into 3 categories: recharging, life-event, & keeping a work-life balance. Just ensure the required work gets done and clear it with your team in advance. You need to take at least two weeks off every year. The average Klue team member takes 2-4 weeks of PTO per year.

⬇️ ⬇️ ⬇️ ⬇️ ⬇️

Lastly, we take potential into consideration. An equivalent combination of education and experience may be accepted in lieu of the specifics listed above. If you know you have what it takes, even if that’s different from what we’ve described, be sure to explain why in your application. Reach out and let’s see if there is a home here for you now or in the future.

We’ve made a commitment to support and contribute to a diverse environment; on our teams and in our community. We’re early in our journey; we've started employee led resource groups, committed to Pay Up For Progress, and use success profiles for roles instead of 'years of experience'. We continue to scale our efforts as Klue grows. We’re proud to be an equal opportunity employer and have dedicated that commitment to our current and future #kluecrew. During the interview process, please let us know if there is anything we need to make more accessible or accommodate to support you to be successful.

All interviews will be conducted via video calls. We work in a hybrid model of WFH (remote) and in-office. We’re excited to meet you and in the meantime, get to know us:

🌈 Pay Up For Progress & 50 - 30 Challenge & Klue Blog

✅✅ Win-Loss Acquisition (2023)

🅰️ Series A (2020)

🐅 Series B (2021)

🏆 Culture, culture, culture!

🎧 Winning as Women & Competitive Enablement Show

🔍 Glassdoor

🐝 About Us

🐥 Twitter

📸 Instagram

☕️ LinkedIn

🦄 Wellfound (AngelList)