Feedback
Follow Us
Publication

“I’m fully who I am”: Towards centering transgender and non-binary voices to measure biases in open language generation

By Anaelia Ovalle, Palash Goyal, Jwala Dhamala, Zachary Jaggers, Kai-Wei Chang, Aram Galstyan, Richard Zemel, Rahul Gupta
2023
Download Copy BibTeX
Share
Download
Copy BibTeX
Share
Warning: This paper contains examples of gender non-affirmative language which could be offensive, upsetting, and/or triggering.

Transgender and non-binary (TGNB) individuals disproportionately experience discrimination and exclusion from daily life. Given the recent popularity and adoption of language generation technologies, the potential to further marginalize this population only grows. Although a multitude of NLP fairness literature focuses on illuminating and addressing gender biases, assessing gender harms for TGNB identities requires understanding how such identities uniquely interact with societal gender norms and how they differ from gender binary-centric perspectives. Such measurement frameworks inherently require centering TGNB voices to help guide the alignment between gender-inclusive NLP and whom they are intended to serve. Towards this goal, we ground our work in TGNB community voices and existing interdisciplinary literature to assess if and how the social reality surrounding experienced marginalization by TGNB persons contributes to and persists within Open Language Generation (OLG). Specifically, we center TGNB voices by understanding their daily marginalization stressors. This informs our OLG harm evaluation design. As such, we focus on evaluating (1) misgendering and (2) harmful responses to gender disclosure. To do this, we introduce the TANGO dataset, comprising of templatebased text curated from real-world text about TGNB individuals within a TGNB-oriented community. We also create an automatic misgendering evaluation tool and measure misgendering across several popular generative models, including ChatGPT. We discover a dominance of binary gender norms within the models; large language models least misgendered subjects in generated text when triggered by prompts whose subjects used binary pronouns.

Meanwhile, misgendering was most prevalent when triggering generation with singular they and neopronouns. When prompted with gender disclosures, generated language contained stigmatizing language and scored most toxic when triggered by TGNB gender disclosure. Our findings warrant further research on how TGNB harms manifest in LLMs and serve as a broader case study toward concretely grounding the design of gender-inclusive AI in community voices and interdisciplinary literature.
Research areas
Tags

Latest news

Work with us

See more jobs See more jobs
Manager, Research Science, Fulfillment Planning and Execution Science - Fulfillment Optimization
US, WA, Bellevue
Have you ever wondered how Amazon predicts when your order will arrive and how we ensure that it actually arrives on at the promised date/time? Have you wondered where all those Amazon semi-trucks on the road are headed? Are you passionate about increasing efficiency and reducing carbon footprint? Does the idea of having worldwide impact on Amazon's logistics network including our planes, trucks, and vans sound exciting to you? If so, then we want to talk with you! At Amazon's Supply Chain Optimization Technologies (SCOT), we are tasked with optimizing the fulfilment on customer orders so that we fulfil all orders worldwide in the most intelligent manner while ensuring Amazon customers get their orders on time. Amazon Fulfillment Planning & Execution (FPX) Science team within SCOT- Fulfilment Optimization group is seeking Manager Research Science with expertise in Machine Learning and/or Optimization and a proven record of leading scientists and solving business problems through scalable ML solutions. FPX Science tackles some of the most mathematically complex challenges in transportation planning and execution space to improve Amazon's operational efficiency worldwide. We own Amazon’s global fulfilment center and transportation planning and execution. The team also owns the short-term network planning and execution that determines the optimal flow of customer orders through Amazon fulfilment network. This includes developing sophisticated math models and controllers that assign orders to fulfilment centers to be picked and packed and then planning the optimal ship method in terms of cost, speed and carbon impact to deliver to the customer. These plans drive downstream decisions that are in the billions of dollars at Amazon Scale worldwide. The systems we build are entirely in-house, and are on the cutting edge of both academic and applied research in large scale supply chain planning, optimization, machine learning and statistics. These systems operate at various scales, from real-time decision system that completes thousands of transactions per seconds, to large scale distributed system that optimize Amazon’s fulfilment network. As Amazon continues to build and expand the first party delivery network, this role will be critical to realize this vision. Your team and tech solution will have large impacts to the physical supply chain of Amazon, and play a key role in improving Amazon consumer business’s long-term profitability. If you are interested in diving into a multi-discipline, high impact space this is the team for you. We’re looking for a passionate, results-oriented, and inventive Scientist who can lead from the front towards developing and deploying ML models for our outbound transportation planning systems. In addition, you will be working on design, development and evaluation of highly innovative ML models for solving complex business problems in the area of outbound transportation planning systems. The position is located at Bellevue, WA, just next to Seattle with beautiful outdoors and great city life. Watch http://bit.ly/amazon-scot to get the big picture. Key job responsibilities As a Manager of Research Science within FPX Science team, you will lead a team of research and applied scientists towards designing and deploy solutions that will likely draw from a range of scientific areas such as supervised, semi-supervised and unsupervised learning, reinforcement learning, advanced statistical modeling, and graph models. You will have an opportunity to be on the forefront of supply chain thought leadership by working on some of the most difficult problems in the industry, with some of the best product managers, research scientists, statisticians, and software engineers to integrate scientific work into production systems. You will bring deep technical expertise in the area of Machine Learning, and will play an integral part in building Amazon's Fulfillment Optimization systems. Other responsibilities include: * Lead a team of research and applied scientists towards design, development and evaluation of highly innovative ML models for solving complex business problems. * Technically lead and mentor the scientists on the team. * Research and apply the latest ML techniques and best practices from both academia and industry. * Use and analytical techniques to create scalable solutions for business problems. * Work closely with software engineering teams to build model implementations and integrate successful models and algorithms in production systems at very large scale. * Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model implementation. A day in the life In this critical role, you will be a technical leader in operations research or machine learning with significant scope, impact, and visibility. Your solutions have the potential to drive billions of dollars in impact for Amazon's supply chain globally. As a science manager on the team, you will engage in all facets of the process from ideation, business analysis and scientific research to development and deployment of advanced models. We are seeking someone who wants to lead projects that require innovative thinking and deep technical problem-solving skills to create production-ready machine learning solutions. A successful candidate is able to quickly approach large ambiguous problems, turn high-level business requirements into mathematical models, identify the right solution approach, and contribute to the software development for production systems. Successful candidates must thrive in fast-paced environments, which encourage collaborative and creative problem solving, be able to measure and estimate risks, constructively critique peer research, and align research focuses with the Amazon's strategic needs. We look for individuals who know how to deliver results and show a desire to develop themselves, their team, and their career. About the team Fulfillment Planning & Execution Science team contains a group of scientists with different technical backgrounds including Machine Learning and Operations Research, who will collaborate closely with you on your projects. Our team directly supports critical functional areas across Fulfillment Optimization and the research needs of the corresponding product and engineering teams. We tackle some of the most mathematically complex challenges in facility and transportation planning and execution to improve Amazon's operational efficiency worldwide and at a scale that is unique to Amazon. We often seek the opportunity of applying hybrid techniques in the space of Operations Research and Machine Learning to tackle some of our biggest technical challenges. We disambiguate complex supply chain problems and create ML and optimization solutions to solve those problems at scale.
Applied Scientist, Artificial General Intelligence
US, CA, Sunnyvale
The Artificial General Intelligence (AGI) team is looking for a highly skilled and experienced Applied Scientist, to support the development and implementation of cutting-edge algorithms and models for supervised fine-tuning and reinforcement learning through human feedback; with a focus across text, image, and video modalities. As an Applied Scientist, you will play a critical role in supporting the development of Generative AI (GenAI) technologies that can handle Amazon-scale use cases and have a significant impact on our customers' experiences. Key job responsibilities - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in Gen AI - Design and execute experiments to evaluate the performance of different algorithms and models, and iterate quickly to improve results - Think big about the arc of development of Gen AI over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports
Data Scientist, Sales Insights, Analytics, Data Engineering & Science (SIADS), Sales Insights, Analytics, Data Engineering & Science (SIADS)
US, TX, Austin
AWS is seeking an experienced, self-directed Data Scientist to support Sales, Strategy, and Operations. They will be responsible for finding new ways of leveraging our large, complex data streams to help us serve our customers in their journey to the cloud. A successful candidate will collaborate closely with business stakeholders, product managers, and data engineers on high visibility and high impact initiatives. They will invent, implement, and deploy state of the art machine learning/AI algorithms and systems to understand our data using tools and techniques such as causal inference models. They will build prototypes and explore conceptually large-scale ML solutions. Beyond mathematical understanding, they have a deep intuition for machine learning that allows them to discover new insights and optimize our sales intelligence offerings. They are able to pick up and grasp new research and identify applications or extensions within the team. They are a superb written and verbal communicator. Key job responsibilities Work with business stakeholders, product managers, data scientists, and engineers to translate business problems into the right machine learning, data science, and/or statistical solutions. Execute every stage of the machine learning development life cycle; researching, developing, deploying, scheduling in production, measuring adoption, improving, and maintaining. Build state of the art causal inference models to help the business understand its key drivers Work with large volumes of structured and unstructured data spread across multiple databases. Design and implement data pipelines to clean and merge these data for research and modeling. Use AWS services (AWS Redshift, S3, EC2, Glue, etc) to deploy scalable ML models in the cloud. Communicate insights to business owners in concise, non-technical language. Examples of projects include: propensity-to-buy prediction and explanation, product recommendation, forecasting, anomaly detection, text classification, generative AI content generation About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. About Sales, Marketing and Global Services (SMGS) SMGS is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector.
Applied Scientist, AWS HealthScribe
US, CA, Santa Clara
Amazon AI is looking for an Applied Scientist to join our science team in the area of Natural Language Processing and Health AI. Our organization develops the science that drives the cloud-based AI services of AWS. Our mission is to put the power of AI into the hands of every developer. We seek to advance the state of the art in machine learning to create services that delight our customers and meet real-world business needs. As an Applied Scientist you will partner with talented scientists and engineers to design, train, test, and deploy machine learning models. You will contribute to innovative features, improve our services based on customer requirements and help maintain a highly scalable data and model management infrastructure that supports cutting-edge research. You will be responsible for translating business and engineering requirements into deliverables and software products. We are looking for candidates who thrive in an exciting, fast-paced environment and who have a strong personal interest in learning, researching, and creating new technologies with high customer impact. Prior domain knowledge in speech, natural language processing, or computer vision is strongly preferred; solid knowledge of fundamentals of statistics, machine learning, and deep learning is required. Candidates should possess strong software engineering skills and several years of industry experience. About the team AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (Iot), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our U.S. Amazon offices.
Applied Scientist, AWS Support
ZA, Cape Town
AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. We are a new team in AWS' Kumo organisation - a combination of software engineers and AI/ML experts. Kumo is the software engineering organization that scales AWS’s support capabilities. Amazon’s mission is to be earth’s most customer-centric company and this also applies when it comes to helping our own Amazon employees with their everyday IT Support needs. Our team is innovating for the Amazonian, making the interaction with IT Support as smooth as possible. We achieve this through multiple mechanisms which eliminate root causes altogether, automate issue resolution or point customers towards the optimal troubleshooting steps for their situation. We deliver the support solutions plus the end-user content with instructions to help them self-serve. We employ machine learning solutions on multiple ends to understand our customer's behavior, predict customer's intent, deliver personalized content and automate issue resolution through chatbots. As an applied scientist on our team, you will help to build the next generation of case routing using artificial intelligence to optimize business metric targets addressing the business challenge of ensuring that the right case gets worked by the right agent within the right time limit whilst meeting the target business success metric. You will develop machine learning models and pipelines, harness and explain rich data at Amazon scale, and provide automated insights to improve case routing that impact millions of customers every day. You will be a pragmatic technical leader comfortable with ambiguity, capable of summarizing complex data and models through clear visual and written explanations. Amazon knows that a diverse, inclusive culture empowers us all to deliver the best results for our customers. We celebrate diversity in our workforce and in the ways we work. As part of our inclusive culture, we offer accommodations during the interview and onboarding process. If you’d like to discuss your accommodation options, please contact your recruiter, who will partner you with the Applicant-Candidate Accommodation Team (ACAT). You may also contact ACAT directly by emailing [email protected]. We want all Amazonians to have the best possible Day 1 experience. If you’ve already completed the interview process, you can contact ACAT for accommodation support before you start to ensure all your needs are met Day 1. Key job responsibilities - Analyze complex support case datasets and metrics to drive insight - Design, build, and deploy effective and innovative ML solutions to optimize case routing - Evaluate the proposed solutions via offline benchmark tests as well as online A/B tests in production. - Drive collaborative research and creative problem solving across science and engineering team - Propose and validate hypothesis to deliver and direct our product road map - Work with engineers to deliver low latency model predictions to production About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Applied Scientist III, Next Gen Developer Experience Science, Q Code Analysis
US, NY, New York
Amazon AI is looking for world class scientists and engineers to join its AWS AI Labs to develop groundbreaking generative AI technologies in Amazon Q, which is an interactive, AI-powered assistant for developers. You will be part of the Q Code Analysis team that works at the intersection of code analysis, logical reasoning and machine learning to build and enhance capabilities, safety and security of AI-powered developer tools in Amazon Q. You will invent, implement, and deploy state-of-the-art algorithms and systems, and be at the heart of a growing and exciting focus area for AWS. Your work will directly impact millions of our customers in the form of products and services that are based on large language models, retrieval-augmented generation, code analysis, responsible AI, and a lot more. You will make breakthroughs that challenge the limits of code analysis, machine learning and AI while collaborating with academics and interacting directly with customers to bring new research rapidly to production. Diverse Experiences: AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture: Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth: We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance: We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work: We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our Amazon offices. About the team The Amazon Web Services (AWS) Next Gen DevX (NGDE) team uses generative AI and foundation models to reimagine the experience of all builders on AWS. From the IDE to web-based tools and services, AI will help engineers work on large and small applications. We explore new technologies and find creative solutions. Curiosity and an explorative mindset can find a place here to impact the life of engineers around the world. If you are excited about this space and want to enlighten your peers with new capabilities, this is the team for you.
Senior Applied Scientist, Sponsored Products
US, WA, Seattle
Amazon is investing heavily in building a world class advertising business and developing a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses for driving long-term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities. Sponsored Products DP Experience and Marketplace org is looking for a strong Senior Applied Scientist who has a track-record of performing deep analysis and is passionate about applying advanced ML and statistical techniques to solve real-world, ambiguous and complex challenges to optimize and improve the product performance, and who is motivated to achieve results in a fast-paced environment. The position offers an exceptional opportunity to grow your technical and non-technical skills and make a real difference to the Amazon Advertising business. As a Senior Applied Scientist on this team, you will: * Be the technical leader in Machine Learning; lead efforts within this team and collaborate across teams * Rapidly design, prototype and test many possible hypotheses in a high-ambiguity environment, perform hands-on analysis and modeling of enormous data sets to develop insights that improve shopper experiences and merchandise sales * Drive end-to-end Machine Learning projects that have a high degree of ambiguity, scale, complexity. * Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production; work closely with software engineers to assist in productionizing your ML models. * Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. * Research new and innovative machine learning approaches. * Promote the culture of experimentation and applied science at Amazon About the team Sponsored Products (SP) is Amazon's largest and fastest growing business. Over the last few years we grown to a multi-billion dollar business. SP ads are shown prominently throughout search and detail pages, allowing shoppers to seamlessly discover products sold on Amazon. Ad experience and market place is one of the highest impact decisions we make. This role has unparalleled opportunity to grow our marketplace and deliver value for advertisers and shoppers. You will invent new experiences and influence customer-facing shopping experiences; this is your opportunity to work within the fastest-growing businesses across all of Amazon!
Data Scientist (L5), CoBRA
IN, KA, Bengaluru
IN Consumer BI Reporting and Analytics (COBRA) team is looking for a highly driven, customer-obsessed Data Scientist who will be responsible for scaling Insights & Science charter across Category & Seller orgs and enable effective business decision making across IN Stores. You’ll analyze large amounts of data, discover and solve real world problems, build science-driven solutions around key projects and, most of all, be an integral part of creating a better customer and seller experience. We are looking for customer obsessed, data driven entrepreneurs to join our growing team and solve some of the hardest problems for our customers. If you want operate at start up speed, solve some of the hardest problems and build a service which customers love, this role might just be the place for you. The Data Scientist is responsible for unlocking business value across IN Stores through advanced analytics (ML, Science/Statistics). The Data Scientist's responsibilities include, but are not limited to the following points: -Collaborate with business stakeholders and product/program managers to innovate on behalf of customers leveraging data science methodologies, and partner with Data engineers and BiE's to design, develop, and scale machine learning models -Use computational methods to identify relationships between business data and outcomes, define outliers and anomalies, and justify those outcomes to business customers -Develop code to analyze data (SQL, PySpark, Scala, etc.) and build statistical and machine learning models and algorithms -Work with distributed machine learning and statistical algorithms to harness enormous volumes of data at scale to serve our customers -End-to-end ownership of operational and technical aspects of the insights you are building for the business, and play a critical role in enabling science based decision making for business - Have excellent business and communication skills to be able to work with business owners to develop and define key business questions and build mechanisms that answer those questions
Applied Scientist , Sponsored Products Ads Response Prediction
US, WA, Seattle
** This position is open to all candidates in Palo Alto, CA, Seattle, WA, NYC and Arlington, VA ** Amazon Ads Response Prediction team is your choice, if you want to join a highly motivated, collaborative, and fun-loving team with a strong entrepreneurial spirit and bias for action. We are seeking an experienced and motivated Machine Learning Applied Scientist who loves to innovate at the intersection of customer experience, deep learning, and high-scale machine-learning systems. Amazon Advertising operates at the intersection of eCommerce and advertising, and is investing heavily in building a world-class advertising business. We are defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long-term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products to improve both shopper and advertiser experience. With a broad mandate to experiment and innovate, we grow at an unprecedented rate with a seemingly endless range of new opportunities. We are looking for a talented Machine Learning Applied Scientist for our Amazon Ads Response Prediction team to grow the business. We are providing advanced real-time machine learning services to connect shoppers with right ads on all platforms and surfaces worldwide. Through the deep understanding of both shoppers and products, we help shoppers discover new products they love, be the most efficient way for advertisers to meet their customers, and helps Amazon continuously innovate on behalf of all customers. Key job responsibilities As a Machine Learning Applied Scientist, you will: * Conduct deep data analysis to derive insights to the business, and identify gaps and new opportunities * Develop scalable and effective machine-learning models and optimization strategies to solve business problems * Run regular A/B experiments, gather data, and perform statistical analysis * Work closely with software engineers to deliver end-to-end solutions into production * Improve the scalability, efficiency and automation of large-scale data analytics, model training, deployment and serving * Conduct research on new machine-learning modeling to optimize all aspects of Sponsored Products business About the team We are pioneers in applying advanced machine learning and generative AI algorithms in Sponsored Products business. We develop customized CTR and CVR prediction models to help shopper feel understood and shop efficiently on off search placements (including product detail pages, thank you pages, home pages, etc.). We develop generative models to understand shopper intent and preference and predict next action.
Sr. Applied Scientist , Sponsored Products Ads Response Prediction
US, CA, Palo Alto
Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. As a core product offering within our advertising portfolio, Sponsored Products (SP) helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The SP team's primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Our products and solutions are strategically important to enable our Retail and Marketplace businesses to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! The Response Prediction team builds machine-learning models and infrastructure to support the Sponsored Products Ads business. Through precise estimation of shoppers' response to ads (e.g. clicks or product purchases), this team helps deliver the most relevant ads experience to shoppers, improves advertisers' ROI, and optimizes Amazon's long-term monetization. The team builds and operates one of the largest ML workflows in WW Advertising, serving Search and Detail Pages. Additionally, it also owns the horizontal ML infrastructure to support various ML use cases - from offline ML pipelines to online model inferencing and model management services. Team video https://youtu.be/zD_6Lzw8raE Key job responsibilities As a Sr. Applied Scientist on this team, you will: - Drive end-to-end Machine Learning projects that have a high degree of ambiguity, scale, complexity. - Perform hands-on analysis and modeling of enormous data sets to develop insights that increase traffic monetization and merchandise sales, without compromising the shopper experience. - Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production; work closely with service engineers to bring your ML models in production - Run A/B experiments, gather data, and perform statistical analysis. - Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. - Research new and innovative machine learning approaches - Publish papers in internal and external conferences
View from space of a connected network around planet Earth representing the Internet of Things.
Have feedback?
Let us know by taking this short survey
Get started
Amazon.com | Conditions of Use | Privacy | © 1996-2024 Amazon.com, Inc. or its affiliates
Follow Us