The Idolmaster Shiny Colors: Song for Prism’s ambitious switch to 3D and the CRI middleware technology that aided the process 

Bandai Namco Entertainment released The Idolmaster Shiny Colors: Song for Prism (hereinafter referred to as Song for Prism) on November 14, 2023. This title revamped the original 2D idol raising simulator The Idolmaster Shiny Colors, implementing 3D characters, rhythm game mechanics and MVs.  

Song for Prism was developed using the audio optimization middleware CRI ADX, the high-compression and high-quality video middleware CRI Sofdec, and the audio analysis and real-time lip-syncing middleware CRI LipSync. In this interview, we spoke with the game’s developers at Bandai Namco to find out how CRI Middleware’s technology was used to make the 28 idols come to life. Inquiries regarding CRI and their products can be made here.  
 

This interview was conducted in collaboration with another game media outlet, GameMakers. Their article offers a more developer-oriented perspective, so for those interested, please check it out here. (Japanese only) 

──Please start by introducing yourselves. 

Yusuke Takayama: 
My name is Yusuke Takayama, and I am the producer of Song for Prism at Bandai Namco Entertainment. I oversee the project’s direction and manage the budget. As this title belongs to the Shiny Colors IP, my role is also to manage things from a bird’s eye view and make sure the game follows the overall direction and sensibility of the IP. 

Masataka Shimono: 
My name is Masataka Shimono, and I am a technical artist. My job is to give directions relating to 3D art, as well as to convey feedback from the illustration team to various other teams. 

Akihiko Ishikawa: 
My name is Akihiko Ishikawa, and I work on the sound team. People might recognize me better if I identified myself as LindaAI-CUE, as I go by in the credits. The sound team currently has four members, and my main responsibilities as the leader are to manage our progress, to respond to inquiries from other sectors, and to do troubleshooting when problems occur. At first, I was creating assets myself, but from halfway through, I was able to leave this to staff and focus on managing the production progress and checking concepts.

──For our readers who do not know about Song for Prism, could you please give us an overview of the game and some of its features? 

Takayama:
Song for Prism is the latest app in the Idolmaster series, and it is the second title in the Shiny Colors brand. The game features two modes: the Produce mode, in which you raise and train the idols, and the Live mode, in which you have the idols you have cared for shine on stage. Like previous titles in the Idolmaster series, Song for Prism lets players become producers who train the idols, put them on stage, and raise their scores through the rhythm game mode. 

──As the producer, what do you think is the most notable characteristic of Song for Prism? (to Takayama) 

Takayama:
In the Idolmaster series, our main priority is to ensure that the players feel the appeal and charm of the idols. The most important part of Shiny Colors is the charming and animated way the 28 idols belonging to the 283 Production agency are depicted. Thus, the development team of Song for Prism have committed their effort to this point most of all, to allow players to fully enjoy the charm of the idols from a fresh new angle, by transitioning from 2D to 3D. 

──When looking at the Idolmaster series as a whole, it feels like Song for Prism takes on new challenges in terms of game design. Was that your intention? (to Takayama) 

Takayama: 
When considering the basic game design of Song for Prism, we were conscious of the fact that even within the Idolmaster series, there were already many titles that offered a rhythm game experience, such as “The Idolmaster Cinderella Girls: Starlight Stage,” “The Idolmaster: Million Live! Theater Days,” and “The Idolmaster: SideM LIVE ON ST@GE!” 

Since there were many rhythm games on the market already, we felt that if we just published another one, we wouldn’t be able to provide players with a new experience. Therefore, we started with the idea of combining the signature Idolmaster mechanic whereby the producer grows alongside the idols with a catchy and intuitive rhythm game mode. 

How Shiny Colors went 3D 

──Is this when R&D started for the title? Looking back at the market, there were certainly plenty of rhythm games out there. 

Takayama:
Before starting Song for Prism’s development in earnest, we went through a period of planning and research. We started out by asking ourselves, “Will putting Bandai Namco Studios’ 3D know-how to use allow us to do justice to the highly acclaimed 2D graphics of Shiny Colors?”  

──How long did it take to complete the first character after starting 3D model production and R&D? 

Takayama:
About three months, wasn’t it? 

Shimono:
Yes, prototyping was finished around the time we made the face and body for Mano Sakuragi and added some motions and soft body animations. We had Mano dance in her Beyond the Blue Sky (full 283 Production) outfit, and that was the start of things. 

We chose Mano because she is the central character of Shiny Colors, but also because her design is the most challenging. In the 2D illustrations, she is drawn with nuance – she has slightly droopy eyes and gives off a different impression depending on the angle from which you look at her. The interesting thing about Mano is her aura, which hasn’t been seen in many previous Idolmaster center characters. This is why I thought that if we can succeed in making her, then we can succeed in making any other character as well. 

──I see, so it took about three months to complete the model of Mano Sakuragi, including the soft body animations. This is when the sound team got involved, I presume? 

Ishikawa: 
Yes, Shimono came to us and asked us, “Please organize the data, because we’re making the test case, and we want to add voices.” I remember that being the first task I got involved in. 

──Were you already using CRI ADX at that point? 

Ishikawa: 
No, not yet. We first fully introduced ADX a few months after receiving the request. It was when we were formally starting production, and the engineers started holding meetings to discuss what middleware to go with. 

Shimono:
I knew that I wanted to do lip syncing first. I’m not talking about lip syncing to music, but rather, I knew that there would be a vast number of lines of dialogue in the event part, so it would be an incredible amount of work if we were to add facial motions individually by hand. At the time, we weren’t using the tool internally at all, but we decided to use it. I also asked Ishikawa for help with this. 

Ishikawa: 
I participated with Ohki, who is responsible for the main BGM of Song for Prism. 

──How many people were involved in development in the end? 

Takayama:
Ultimately, we ended up with a development team of more than 100 people. 

Shimono:
Our general Zoom meetings, which include asset creators and engineers, have become very crowded. 

Takayama:
I think the breakdown of our team is about 40% asset creators, 50% engineers, and 10% leaders and planners. 
 

Making the 3D models look like the 2D illustrations came to life 

──Please tell us about what differentiates Song for Prism from the Enza version of Shiny Colors and other Idolmaster titles in terms of basic concept?  

Takayama:
Just as I described at the start of the interview, the gameplay concept we had for Song for Prism was for the player to become a producer. As the producer, the player chooses the actions and schedules of the idols – teaching and guiding them to improve their abilities. Once they have been trained, the idols can then perform lives, showing the producer the fruit of their efforts. That is the experiential value we aim to give players with Song for Prism. 

As for graphics, our main point of focus was how to fully convey the appeal of the original 2D illustrations and artwork, which players had praised so highly, in 3D. We wanted to maintain the delicate lines and fine colors of Shiny Color’s artwork in 3D, as if the illustrations just came to life. 

──So, you gave staff directions to make it look as if the 2D illustrations were moving”? 

Shimono:
That’s right. I write that every time on documents such as project proposals. But frankly, I think the illustrations are so beautiful that what we’re doing is not nearly good enough yet… 

Ishikawa: 
I still remember the sound team’s first meeting, when we talked about whether the BGM would be used as dramatic accompaniment, whether we should make the melodic lines stronger or weaker, and whether we should have the BGM support the scenario or build up atmosphere. I considered these to be major points that differentiate Song for Prism from other Idolmaster games. 

I feared from the very beginning that if the melodic lines of the background music were too strong, things would start sounding cluttered. Ohki, whom I mentioned earlier, was also very conscious of this, so we worked hard to create tracks that serve well as background music. 

Background music plays in various scenes of the game, including the Produce Mode and the main home screen of the game, but the common idea we had for all scenes was “maintaining a proper sense of distance.” For example, during game scenarios, the music goes to the background of the scene, letting the story shine. On the other hand, on the home screen, the music becomes more straightforward, and that adds some variation to the scenes. 

──By the way, how many tracks are there in the game? 

Ishikawa: 
When the game was first launched, it had almost 70 tracks. Most of the background music tracks are 60 to 90 seconds long, but there are some exceptions. 

For example, when we really want dynamism in a BGM track, we might want it to be a cohesive two- to three-minute piece. In order to make sure such tracks can be used in the game, we asked CRI to add a function that can add markers, so that songs can be played from their middle sections. This allows us to loop different parts of one longer track. 
 

Making sure the 3D graphics don’t hinder the rhythm game experience 

──What points did you have in mind when creating the sounds and graphics for the rhythm game mode of Song for Prism? 

Shimono: 
One big point was not letting the graphics interfere with the rhythm game. I think some players would rather turn off the 3D graphics and play against a 2D background, but I would be a bit disappointed if they did that. Basically, I wanted people to be able to play the rhythm game while enjoying the 3D graphics, so I made maintaining the frame rate our utmost priority. After all, if the master score is choppy even for a moment, it will be detrimental to the playing experience. Rhythm games can be impacted more severely by frame rate than action games if made poorly, so our engineers and asset makers are working extra hard in order to maintain the quality of the game in this respect. 

We actually use different videos for the music video viewing mode and the rhythm game mode. In order to maintain the frame rate, we reduced the weight of effects in the rhythm game mode, making shadows and reflections lighter. In this way, we have made sure that there is no big difference in appearance, but have also taken some measures to ensure a stable frame rate. Speaking of details, we made it possible to manage the rhythm game’s musical score-related user interface and the frame rate of 3D model movements separately, so when there is a slowdown in processing speed, the 3D frame rate is lowered first. We have also continued to make processing improvements following the release, so Version 1.2.0 runs a bit faster. 

──So, maintaining 60 fps is a must for you? 

Shimono:
Yes, it’s a rhythm game, so we are committed to quality in that area. With so many devices with different specifications out there, it’s difficult to guarantee 60 fps. As such, we worked on allowing detailed option settings so that the rhythm game mode can be played smoothly on as many devices as possible. 

──How did you work on the production of the 3D live scenes? Can you tell us about your workflow and what kind of direction you had in mind for the look of the concepts? 

Shimono: 
We begin by getting informed about the concept and direction of the song, and after we have established the production flow and stage design to some extent, we have Takayama take a look at it. Once we are greenlit, we start by designing costumes and producing models, and then we do motion capture in consultation with our choreographer. After that, we brush up the motion, create camera work, assemble the live scenes, add lip syncing and effects, and at last, our work is complete. 

──So the song really comes before anything else. 

Takayama:
That’s right. We start from existing songs, while consulting with Bandai Namco Music Live, the company that makes the CDs. Then, for songs being produced specifically for Song for Prism, we sometimes make requests to Music Live, such as “we want it be on the difficult side rhythm game-wise.” From there, I consult with Shimono and Ishikawa on technical matters. 

──(to Takayama) So you basically do overall supervision of the whole project. (To Shimono & Ishikawa) Do you have any songs or live productions that you liked in particular?  

Shimono:
I always think that the most recent music videos are the best. The facial expressions of the characters, the effects, and the camera all keep leveling up at a rapid pace. The latest song (as of the time of this interview) that we made in order of production is “Hanamuke no Hanataba,” and it is my favorite so far. I also think that it is our highest quality song to date. 

Takayama: 
Yes, the music videos keep making such rapid progress. We have also learned to portray complex formations, with three idols crossing each other’s paths in “Hanamuke no Hanataba” – the camera is able to capture the movements, and the choreographer can be more ambitious, so it produces a synergistic effect. I’m excited for more bold challenges from Shimono in the future. 

Shimono:
We also have several effects artists, and in their workflow, they engage in friendly competition with each other, saying, “Look what they managed to do! I’m going to do something even more awesome!” When we create a good music video, it boosts morale and motivates us, so I am happy when we can get excited about our work in that way. 

──Who composes the musical scores for the rhythm game part? 

Shimono:
There is a musical score composer who composes the scores. They are on the planning side of things. 

Ishikawa: 
Sometimes the composer asks me questions. For example, “What instruments and phrases should I use as a basis to compose this musical score?” 

Shimono: 
The musical scores are handled by a professional experienced with rhythm game musical scores. For example, they even added the text “SOS” to the score for “SOS” and added a note shaped like the number 1 to the part “No. 1” of “Yumesaki After school.” 

Determined not to lose to the latest 3D models 

──I think that a 3D version of Shiny Colors was also a long-cherished wish from fans. How did you approach production in this respect? 

Shimono:
At first, I was worried about whether it would actually work out. I mean, how many polygons would it take just to animate Kogane Tsukioka’s hairstyle! (laughs) 

──The still frames of the Enza version are beautiful in a way unique to 2D illustration, aren’t they! 

Shimono:
I still don’t think we can beat them. (laughs) 

We developed Song for Prism knowing that it would not be able to run on the mobile devices of the time, but with the idea that it would run on future devices.  

If I’m being really honest, we started out only thinking of PC performance. Then, as development went on, games quickly came to have high quality models, and VTubers with complex models started appearing. Looking at those models, I really didn’t want to lose to their level of expressiveness.  (laughs). 

──By the way, how are the character models in Song for Prism regulated? 

Shimono:
It changes depending on the costume, and the distribution of polygons differs between five-member-units and two-member-units. But on average, we use about 15,000 polygons for the face and hair, 3,000 polygons for hair accessories, and 30,000 polygons for costumes. We use two to four textures of 2,048 pixels square, four to five textures of 1,024 pixels square, four to six textures of 512 pixels square, and other textures of several kilobytes. The textures are mostly used for costumes. 

──About the Story Commus that appear in the event part at the end of scenarios, are these for pre-loading? 

Shimono: 
Yes, that’s right. The memorial music video that comes after the Produce Mode ends has many inserted cut scenes in addition to the live scenes, so all of those need to be loaded. 

This is also connected to memory restrictions, and it really depends on the number of members in the unit. Each five-member unit requires five sets of costumes and five sets of plain clothes, as well as tracksuits, and all of these must be loaded. We determine the processing size of each character based on all of this. The two-member unit SHHis has more leeway to work with than other units, but we cannot let the processing drop during the rhythm game part, so it’s not like we can use a ton of resources on them either. 

──It’s impressive that the specifications have not changed from beginning to end. 

Shimono: 
While we are still working on optimization, I am glad we were able to focus on maintaining quality during production. 

The role played by CRI’s middleware 

──Next, could you please tell us about how and where you used CRI middleware in Song for Prism’s development process? 

Shimono: 
We used CRI ADX for everything related to sound, and CRI Sofdec for video sections. Also, we used CRI LipSync for lip syncing. Since we used past assets from the Idolmaster series for some parts of the game, these tools made a big contribution in facilitating sound production flow 

──You have been using CRI ADX for quite a long time, right? 

Shimono: 
That’s right. The team members in charge of sound implementation and engineering both find it easy to use, and they have accumulated many years of know-how. Song for Prism is a live-service game, so any problems with maintainability are sure to cause trouble. Therefore, our idea was to use middleware that has a proven track record. 

Ishikawa: 
When working on sound, the profiler function of CRI Atom Craft* was extremely helpful. We often discover missing material during project development, but the profiler function has a screen design which facilitates understanding of sound, so it was very helpful while debugging in the latter half of the project. 

*CRI Atom Craft is the name of CRI ADX’s specialized audio authoring tool. 

Shimono: 
If we hadn’t introduced that middleware, the development period would have been prolonged. We used CRI LipSync for lip syncing. LipSync Library analyzes audio in real time for visual novel sections such as the Story Commus, as well as idol streams; and LipSync Tools analyzes audio in advance for the live performance parts. 

Takayama:
If I recall correctly, we wanted to use real-time lip-syncing during the real-time Miki Hoshii stream, and I believe that is where we first introduced the tool in the Idolmaster series. 

Shimono:
Yeah, that was definitely a big touchstone for us. 

──Are there are any parts for which you animated mouth motions by hand, without using CRI LipSync? 

Shimono:
There was nothing major done manually from scratch. However, the facial motions were not done simply as analyzed (by lip sync analysis). 

In the early days, we did motions simply as they were analyzed, but some characters were difficult for analysis. It might have been better if we were able to make use of CRI LipSync’s adjustment settings, but we didn’t have full command of the functions at the time, so we would analyze the motions in Unity and then bring them back to Maya to make adjustments. 

Innovations made for optimization 

──The character lip syncing in Song for Prism greatly improved from the demo version. What kind of innovations and improvements did you make? 

Shimono:
The most obvious thing is that we took what we had made in 24 fps and remade it in 60 fps. The fact that the real-time characters were moving at 24 fps was more disconcerting for users than we had expected, so in order to recreate them in 60 fps, we had to drastically reduce the processing load. When we set the lip syncing to 60 fps, the lip sync analysis results were very different from what we got at 24 fps, so we had to rearrange all the groundwork to be able to include the results. 

Loading also became a lot faster. Our engineers worked hard to make downloads possible in the background. By changing our video codec to VP9, we managed to greatly shorten download time. I think the storage taken up was reduced to about 1/3. Sofdec supports VP9, so we were able to do codec changes more smoothly. 

Takayama: 
We also reworked the music videos quite a lot. For example, we revisited parts with too many cuts, sped up the camera work, improved parts where you can’t get a good look at the idols’ faces, as well as the overall coloring. Shimono and the music video team did a lot of research on these aspects so that players can enjoy the charm of the idols, and in the end, we reworked almost everything. 

Shimono: 
We have also brushed up all of the models. When looking at the idols from the producer’s perspective, you are looking down upon them, so we adjusted the balance between the back of the head and the face of each idol to prevent any discrepancy with the impression you get in the Enza version, where you view them from the front. 

──Please tell us about any methods or know-how that you used for optimization. 

Shimono: 
We didn’t do any reductions of the models themselves, but we have reduced textures a lot. One of the most important details was that we went from having skin textures for all the idols to having a single file with a system that allows for colors to be changed. 

──I see. As the producer, did you have a minimum level of quality that you absolutely could not compromise when compressing for optimization (to Takayama)? 

Takayama: 
The most important thing with mobile apps is to be able to reach the function you want as soon as possible, so ensuring this goes above everything else. Then, in the case of VN/story sections, I think there is a certain required quality level that is necessary for players to be able to immerse themselves in the scenario and the idols’ expressions. On the other hand, in the rhythm game mode, players are focused on the notes, so they won’t mind so much if the shadows and effects are reduced compared to the music videos. I believe that different compromises can be made depending on what the players expect at any given time. 

Shimono:
In terms of graphics, I would share the optimization details with Takayama, and if there were any points of concern, I would have them fixed – we communicated closely about these things. If he said he was concerned that the ceiling lights were a bit choppy, or the cookies on a plate in the office kitchen looked flimsy, I would get them fixed, and so on. 

──Thank you very much. I expect that it is important to distinguish real time and pre-rendered graphics when optimizing. What was your standard in Song for Prism? 

Shimono: 
The gacha screen and the opening video are pre-rendered. We want the gacha to be easy to operate, so we use pre-rendering to make the loading time as short as possible. If we did it in real time, the game would have to load a lot of assets, which would take a long time. 

The opening’s loading speed is another issue – with that many cuts, and with all the idols present, we had to use a pre-rendered video because of the sheer amount of data. However, we made every single cut in real time and rendered them in high resolution in Unity. 

The sound design of live scenes was propped up by middleware 

──Could you also tell us about the live performance scenes? How did you do the sound design in terms of having different vocal lines? 

Ishikawa: 
We didn’t do anything too elaborate, but we went through a process of trial and error with the engineering team on how to accommodate so much data. Our first idea was to throw everything into a single cue sheet, but if we did that, then it would take up a tremendous amount of storage. Therefore, we decided to basically make cue sheets for each vocal track, so that when the character composition was decided, we could load only the amount that was necessary. 

──So you made it so that only cue sheets for selected characters are brought over, rather than all of them. How is the mechanic of adding sound effects etc. to the live performance controlled? 

Ishikawa: 
Basically, sound effects play when the player taps the screen, so we really made it in the standard fashion. We prepared five different sounds that suit the skills and preferences of each player. I’d like to keep an eye on players’ reactions to this and add new features at some point.  

──In the beginning of the interview, you said that you implemented a function for adding markers so that you can play tracks from their middle sections. Do you have any examples of sound-related mechanics that you were only able to implement with CRI’s technology?  

Ishikawa: 
The markers are definitely one example, and with the background music on the home screen, we slightly switch the sound played each time the scene changes. Using the same technology that we used in The Idolmaster: Million Live! Theater Days, we made it so that the background music can naturally switch to match the scenario. 

──You also used SonicSYNC for Song for Prism’s development, but how did you come to adopt this software? 

Ishikawa: 
Our sound team had received several presentations in the past, and after trying out the demo, we found it satisfactory. However, we were undecided as to whether or not we would actually use SonicSYNC. This is where our engineers suggested that we try it, and after a test run, we decided to adopt it. 

Shimono:
There are various middleware that are useful for their individual features, but only CRI’s middleware provides us with everything we need in one place. Another important point is that we have accumulated experience with them. After all, when we use a new tool, our engineers must learn how to use it from scratch. 

──It must be reassuring to know that all the tools are provided by one company, and that they provide support in Japanese. 

Shimono:
Yes, and it also helps that they respond properly to our inquiries. 

Ishikawa: 
I was very happy with how quickly they replied to our questions, and how certain they were about whether they could or could not do something. I think the most important thing for me was that even when they could not respond immediately, they would tell us when they would be available to help. We were able to share their answers among ourselves and discuss when to introduce certain solutions during the development process, which made the work go very smoothly.  

──Could you please tell us frankly about your impressions of using CRI products? 

Shimono:
I think it’s really amazing that the sound-related workflow can be unified across various functions and devices. Honestly, when the app crashes, it often happens because of the CRI middleware part crashing. This could also be due to us not using the functions the way CRI had intended, so we report issues and work things out one at a time. It’s times like these that their support division provides us with very helpful information. They respond to inquiries very quickly, and even when they are unable to respond, they let us know how long it will take until they are able to respond, and I really appreciate that. 

Ishikawa: 
Another thing that helped a lot was the high scalability. In order to improve work efficiency, we use Atom Craft’s scripting function to make tools internally, so that we can improve our work speed. 

Takayama:
Looking back on the project, we actually used CRI’s products in many different areas of development, and this is because we have been a long-time patron of theirs. Many of our staff are familiar with their tools, which further helps development go smoothly. We intend to keep using their tools in the future. 

──Thank you very much. 

The Idolmaster Shiny Colors: Song for Prism is available for the PC (DMM Games), iOS and Android. CRI are accepting inquiries via their official website. 

Interviewer: Daiki Kamiyama 
Writer, editor: Aki Nogishi 
Photographer, editor: Ayuo Kawase