Challenges and Results

Challenges

  1. Because the work frequently involves personal information, security and governance reviews for new tools were stricter than in other departments, and exploring generative AI kept getting pushed back
  2. The work calls for hospitality-driven customer service that goes well beyond routine handling, so operators took a long time to become fully capable, and low retention was also a challenge
  3. Inquiries from drivers and dealers together reached about 1.5 million a month, and analysis to understand variation in response quality and recurring issues could no longer keep pace with the growing volume

Results

  1. Making the “ideal state in three years” visible created a shared understanding across the company of a direction for generative AI that had been abstract
  2. A PoC confirmed that generative AI can technically classify a focused set of inquiries, providing a concrete foothold toward production rollout

Before

Stalled progress on generative AI, and the time it takes to train operators

The inquiries this call center handles go beyond simple administrative procedures. They call for a level of hospitality that could be described as true “omotenashi.” As a result, operators needed considerable time before they could work independently, and low retention had long been a challenge.

In addition, because the work frequently involves personal information, introducing a new tool meant stricter security and governance reviews and requirements than in other departments. The evaluation itself took time, and exploring generative AI had repeatedly been put off. Meanwhile, inquiries from drivers and from dealers together reached about 1.5 million a month. Analysis was needed to understand variation in response quality and recurring issues, yet the work still had to be done by hand. The speed of analysis could not come close to keeping up with the growing volume, and the burden on the front line kept increasing.

Figure: Three challenges facing the call center

Redesign

Picturing the ideal state in three years and building on proof points

The project set about 10 use cases as items to test, including automatic FAQ creation. As the first step, ITSO spent about three months developing a demo of a call support app. Designed around the screen an operator sees during a call, it transcribes the conversation in real time, detects its context, and suggests on the spot the kind of guidance the operator should offer. For example, when a customer talks about traveling to a destination, the AI checks the weather forecast for that area and makes a thoughtful suggestion such as bringing an umbrella. When the operator wants to explore a topic further, the AI automatically runs a web search and presents a summary of the results. Technically, the app is built on a GPT model integrated with a search API.

The demo was presented at an internal results briefing and was highly rated, with more than 80% of survey respondents saying they had high expectations for its future use. Turning the “ideal state in three years,” until then only a vague vision, into something people could actually see created a shared understanding within the company, which was a major step forward.

From a company-wide proposal to a classification PoC using an on-premises LLM

Encouraged by this response, ITSO spent the next four months or so proposing the adoption of generative AI across the entire call center.

ITSO then spent about five months on a PoC for an app that classifies inquiries by category using an on-premises LLM. Because the work frequently involves personal information, an environment that could process data without sending it outside was a requirement, so an on-premises LLM was chosen instead of a cloud service. Analyzing all 1.5 million inquiries a month by hand is unrealistic, and the quality of response required differs by product and service lineup. The first need was to sort inquiries into broad categories: those that should be analyzed and those that need no analysis. For the validation, ITSO narrowed the scope to inquiries that are few in number but high in importance. Using data that contact center staff had sorted and uploaded by hand, ITSO tested the AI’s classification accuracy and secured a solid technical foothold toward production rollout.

Organizational hurdles

Two major hurdles stood in the way of production. The first was unifying cooperation across several departments: front-line operators, the strategy department, and the infrastructure department. The second was that many existing systems had been built by outside vendors, so connecting them with AI required coordination with multiple vendors and existing infrastructure, in some cases including data kept on paper or in legacy systems. This took far longer than expected.

After

A foundation of shared understanding

The greatest outcome of this project was taking the “ideal state in three years,” which had only been discussed in loose terms as something that would be nice to have, giving it visible form as a demo, and turning it into a shared understanding across the company. In addition, the PoC confirmed that generative AI can technically classify a focused set of inquiries, providing a concrete foothold toward production rollout.

The ultimate goal for three years out is to connect seamlessly with the existing customer data system, centralize the data, and automate both operator support and inquiry analysis. The project continues to move steadily forward while bringing stakeholders on board.

If your organization is finding it hard to move forward with generative AI because of the personal information you handle, or if analyzing a large volume of inquiries has reached the limits of manual work, we hope this case offers useful insights.