Job Information
Microsoft Corporation Software Engineer II in Bangalore, India
Microsoft's vision for Azure Machine Learning (ML) centers around democratizing ML and ensuring its accessibility to all enterprises, developers, and data scientists. We are seeking individuals to join our team entrusted with the responsibility of serving all internal and external ML workloads. With our current efforts, we are already catering to billions of requests daily, encompassing the most cutting-edge scenarios and models throughout the company.
As a member of the Inference team, you will contribute to the development of the next generation of model serving. This includes hosting OpenAI models such as ChatGPT, as well as scaling model hosting for Bing and Office, tackling numerous captivating challenges at the intersection of AI and Cloud. We are actively seeking a highly skilled Software Engineer who possesses a profound passion for designing and constructing exceptionally reliable and available platforms, capable of supporting model inferencing on a massive scale.
In addition to platform development, you will be tasked with addressing high throughput/low latency scenarios and spearheading performance optimization capabilities. This position provides a unique opportunity to thrive in an environment that fosters innovation, fosters collaborative teamwork, and upholds the pursuit of excellence, all in alignment with Microsoft's mission.
#AIPLATFORM
Responsibilities
Engage directly with key partners to understand state-of-the-art LLMs and Diffusion models, run them at scale in performance and cost effective manner
Leverage latest hardware stack technologies improvements in CUDA, infiniband and fast-moving software stack to deliver best of class inference
Anticipate, identify, assess, track, and mitigate project risks and issues in a fast-paced start up like environment
Motivated to build constructive and effective relationships and solve problems collaboratively
Support production inference for core AI scenarios on one of the largest GPU fleets in the world
Qualifications
Required Qualification:
B Tech or M Tech in computer science, engineering, mathematics or a related field, or equivalent industry experience
4+ years of software development experience
2+ years of software development experience focused C/C++ and/or Python development
Knowledge and experience in OSS, Docker, Kubernetes, Python, GOLANG programming languages
Preferred Qualification:
Practical experience hosting and running large scale machine learning models in enterprise grade applications
Knowledge of LLM/Diffusion model architectures e.g. GPT and Stable Diffusion
Experience in building enterprise grade applications in C++, Python
Experience in developing low latency systems
Experience in developing and operating high scale, reliable online services
Good communication, collaboration skills and a great team player
Experience working in a geo-distributed team
Understanding of parallel algorithms for communication between GPUs, familiarity with related libraries and frameworks such as DeepSpeed, PyTorch Distributed, Horovod, Megatron, MSCCL, NCCL is a plus
Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (https://careers.microsoft.com/v2/global/en/accessibility.html) .