AI Powered Gaming Assistant Gaming Assistant Video games

Far Cry 6 with Fridai – Making an accessible game even more inclusive

Quick summary

  • Ubisoft is leading the gaming industry with accessibility features and Far Cry 6 is another great example of this. 
  • The accessibility market has billions of dollars of potential.
  • Voice integration combined with NLP is the next step for inclusive gaming, and Fridai is ready to boost accessibility for projects. 
  • You can already download the next-gen accessibility tool for gaming from the Microsoft Store. It is called Hello Fridai.

Far Cry 6 – a new milestone for accessibility in gaming

Ubisoft continues to elevate accessibility in gaming with its blockbusters. We already saw remappable controls and 3rd party app integrations such as the Tobii Eye Tracker in Assassin’s Creed: Valhalla for example, and fortunately FC6 is no exception to this trend. 

By just simply looking at the very first screen that welcomes the players in the game you can tell that they are taking the matter of Accessibility very seriously. Presets such as hearing, vision, motor, cognitive, motion, colors already give the players a headstart to tailor the experience to their needs and it is not an overstatement to say that you can customize EVERYTHING in the controls menu. The accessibility options are truly robust, and if you want to know more about them, there is an excellent summary on written by Antonio I. Martinez.

The market speaks for itself

In case you are wondering, what drives the industry to make inclusivity a big thing these days, just take a look at these numbers: 

  • USD 21 bn left on the table (US market) – Differently abled gamers are ruled out to play many games, which is a social problem for them, and a market coverage problem for the publishers 
  • Gamers want to play no matter what – ~10% of the population has a form of impairment that would stop them from playing games. ~92% of gamers play games in spite of their impairment.

It is not just simply fair, it also makes a lot of sense from the business perspective. (

Next-level inclusivity with the power of Fridai

Just like Ubisoft, at Fridai we also believe that every game should be playable and enjoyable for everyone. 

We already launched voice control support for Red Dead Redemption 2 back in August and we received a ton of positive feedback from our users and acknowledged experts in the field such as SpecialEffect and Game Accessibility Nexus. We did not hesitate to use these learnings to create a voice control support for Far Cry 6 as soon as it hit the shelves in early October. 

The goal was simple: 

A person who has at least access to a headmouse and can press at least 1 button should be able to play Far Cry 6.

As we already discussed in our Red Dead Redemption 2 article, there are several tools available for voice control, but setting them up can not only be cumbersome, but usually, they exclusively understand strict-matches. This means it understands the commands only if the speech to text picks it up perfectly. If it doesn’t, nothing happens, or worse: you end up dead because of the wrong input.

With Fridai, we tackle this challenge by utilizing advanced NLP technology and our unique in-built scoring system for intent recognition. 

The user speaks, the speech-to-text engine picks it up, NLP does its magic, and based on a humongous amount of training data, corrects the content – if necessary. But this is only one part of the magic. 

NLPs are still imperfect, and misunderstandings are inevitable. Fridai is built the way to be able to get the intent right even if the input text is just partially correct. Our unique scoring system evaluates the confidence indexes of the intent options and chooses the one that the user is looking for in that specific situation. 

This is how it is possible with Fridai to enter a vehicle in a game with a voice command even if the Speech-to-Text engine picked up “and their vehicle”. Genius. 

But how does it play? 

What’s next?

We cannot emphasize enough how important it is to Gaming and the Accessibility Community that industry-leading companies such as Ubisoft make true efforts for increased inclusivity. Everyone should have the opportunity to experience these games and have a great time doing so. 


And this perfectly resonates with the vision of Fridai:

Bringing a lifestyle companion to every gamer in the world, eliminating all barriers.

As we are working hard on perfecting Fridai’s voice control features, we are also exploring opportunities to work closely with game developers on their upcoming titles. If you know about a project where we can help with accessibility, don’t hesitate to contact us. 

Take Fridai for a spin!

If you like what you read so far, go ahead and download Fridai from the Microsoft Store. We would love to hear your feedback, so don’t hesitate to join our Discord group and hit us up! 

AI Powered Gaming Assistant Gaming Assistant Video games

Making Red Dead Redemption 2 accessible with Fridai, the voice assistant

Accessibility and making games more inclusive has always been a high priority for our team, ever since the first community engagement we had about how Fridai was really helpful for our members to access information for their favorite games. We even had a documentary video made about our approach as part of the Digital Defenders Series, if you are interested you can watch the video here:

Voice access for games – our journey

The underlying technology of Fridai has always been voice, the way we process audio streams and use them to control the games and provide guides or strategies made our software unique in the industry. 

This ambient technology however is capable of so much more than we initially thought. The original pain point we set out to solve was to provide a way for gamers to eliminate distractions from their gameplay, access guides and walkthroughs without opening any other apps, or looking at other screens. While it remained as a selling point of Fridai, the main focus has been shifted towards an even bigger pain point we know in gaming: accessibility.


The current state of gaming in terms of accessibility

After thorough research we have found that even though games today include accessibility settings, they are most of the time far from covering the angles gamers really need. We have seen games where the only accessibility setting you could access was the subtitle settings, if you wanted to increase or decrease the size of the fonts or their colors. There was no mention of customizing keybinds, using a special controller or even providing a different way of interacting with the actual game.

Going even deeper into the numbers (US market) we can see that around 10% of the population actually has a form of impairment that would stop them from playing games. More interestingly 92% of gamers play games in spite of their impairment. The question here of course is which games can they play?

There are currently several solutions for disabled gamers to get more information about games, if they can play them or not. There are game review sites, coming from the community who test games from the accessibility perspective and after reading through them it is possible to see if a game is playable according to one’s disability or not. One of many problems with this is that you get to read a review after the game comes out, so in case you wanted to pre-order any of the games you could but risk not being able to play it. There is a great article about game previews and the lack of these actually affecting disabled gamers by our friends at Can I Play That, I recommend you to read that, too.

We have come to a conclusion that in many cases accessibility feels like an afterthought for game studios and we simply can not accept that not every game can be played by anyone.

Our contribution: NLP powered voice control

The way we approached this problem was setting a north star or a vision for our features: a person who has at least access to a headmouse and can press at least 1 button should be able to play any game they like. This might look like a limited vision, however for the first set of features we wanted to develop this set the bar very high already – since we are talking about voice technology and not neural links or eye tracking – just yet.

We know three aspects of Fridai that can make it a very powerful assistant when it comes to accessibility tools:

  1. Fridai can understand many accents and can figure out what was said even with very noisy backgrounds, pays attention to mispronunciations and gets the intent right even if the sentence being said was just remotely accurate
  2. We already have applied voice control for FIFA 20 as a multi tasking tool, where you could change players by voice
  3. Voice provides a new interaction layer when it comes to the players and their PCs or consoles and the only thing you need is the ability to speak, therefore it could help a huge amount of players.

Naturally at first we took a look at how players use current voice solutions, to create macros, setup voice recognition and so on. We had interviews with many players who were using any of these to really dig deep into the user experience. What we have found is that current voice solutions use a technique that can be described as strict match, so only if an exact word is understood will the associated action activate. In comparison, with Fridai, we could activate one action for a wider range of expressions, so as an example, you will not need to map “mount, mount horse, mount the horse etc.” one by one, once one of these is said Fridai can understand the intent of the player and work right then.

Once we saw how the natural language processing algorithm we build can aid users and can improve their gaming experience, compared to what services they use today, we set out to work out the weaknesses we had – again compared to the currently available solutions. One of these being the fact that Fridai needed to be activated using a wake word – so whenever you say “Hey Fridai” the AI wakes up waiting for the command. Now while this provides a hands-free solution, it actually can take a couple seconds to activate the action, hence might not provide a great gameplay experience. So we added keypress wakeup to Fridai, as a result you can quickly wake it up by pressing one key. (Remember our vision about only having to access one key and a head mouse? Now this is part of it.)

Once both of these were completed we set out to find a game that a lot of people want to play, but were previously unable to thanks to the complexity of the controls.

For a better explanation of how NLP works in games, here is one of our videos for the Witcher 3. It shows how different expressions activate the same intent.

SpecialEffect and Red Dead Redemption 2

Our approach about picking a game was simple: we asked ourselves, which game is the one that tons of people would want to play but they can not for some reason. So we started reaching out to accessibility focused organizations, to understand their needs and ask them to provide feedback about Fridai from their users’ perspective.

This is how we’ve found the team at SpecialEffect, who responded to our query right away. We focused first around collecting their feedback about voice control and current solutions out there and we got a great overview of the market and needs. Even during our first call, we saw that bringing our NLP into the world of voice control, could provide the seamless interface to games that we set out to build and more importantly what service users of SpecialEffect were looking for. We have settled on building the first version of voice control powered by Fridai for Red Dead Redemption 2, as that is a game that is as complex as it gets from the control perspective and many in the audience of SpecialEffect were looking forward to a solution that enables them to play this game.

Now it was about time we built Fridai into Red Dead Redemption 2, with the goal of making it playable and enjoyable through voice access.

Implementing voice control for Red Dead Redemption 2

We already have laid the foundations for supporting voice control with Fridai, therefore it did not take us a long time to implement the actual voice based game control into the game. The first task we had to do was going through the keyboard settings and current keybinds and assign intents to each one of those.

Once it was completed, we started playing the game and constantly followed the instructions on the screens during mission and free roam gameplay, so we could capture the actions and understand what users would say as voice commands while playing. As an example, when it comes to the player interacting with the horse, we added the following intents that Fridai would later recognize:
Mount horse, dismount horse, call horse, hitch horse, brush horse, feed horse etc.

All of these commands activate a corresponding intent and thanks to our NLP engine even different variations of these work when players are speaking to Fridai, so we completely removed the necessity of having to remember exact commands.

Even though we received constant feedback from SpecialEffect as we progressed with development, we made time every week to check-in for a live testing of Fridai. Each of these sessions resulted in more patterns and expressions we needed to teach Fridai, however we got closer and closer to a solution, when it finally looked like this:

As a summary, now we are proud to say you can now play through most of the story missions and do free roam using only your voice. You only need access to a piece of hardware that can control your camera – but we will fix this soon, too.

AI Powered Gaming Assistant Gaming Assistant Video games

Rethinking gaming guides with GPT3 – Fridai and the Witcher 3: Wild Hunt

The basic idea behind Fridai has always been to build an AI that behaves like a friend sitting with you while playing your favorite games – because we all remember how much fun it was to play together with our friends as kids. With an AI friend, of course, the experience will be different, but if we can make even a slight resemblance of how that felt, we will have built a truly amazing helper for games.

On the user experience side of this, our goal is to help everyone who enjoys gaming access information in a different way, without having to Alt+Tab out or even look at other screens. The first versions of the assistant delivered on this promise, however as we wanted to add more and more games, we seemed to have hit a barrier in how we could scale it efficiently to other games.

On our journey to find ways to accomplish a constantly improving, more human-like Fridai, we needed the answer to 3 questions:

  1. How can we create a better personality for Fridai?
  2. What is the most efficient way to keep Fridai’s knowledge updated?
  3. Can we make Fridai provide more native advice while gaming?

In this post we will only discuss our answers and methods to the last 2 questions, as they are related to the engine we’ve built and its connection to GPT3.

This post is not going to cover GPT-3 and how it works, so feel free to go to or read the actual whitepaper here

Ever since the first use cases of GPT-3 I have seen demos for, I’ve been thinking about how to apply the technology to reimagine gaming walkthroughs – and more importantly how gamers can access them today. So let me just insert a little sidetrack here, to explain how I think about game guides.

The idea started from a personal experience and problem, where whilst playing games, I actually enjoyed looking up guides and doing research on certain points (especially items and builds) – to be honest I needed to do this more than I can confess.

From all the content out there, I have found two types to be actually useful – longer websites where I can see the guides for my questions and of course videos with playthroughs. I am not going to go into explaining how much I hated having huge ad banners on the sites or even the fact that I had to wait 15 seconds even for a video to actually get started, we all know this (except of course, Youtube Premium subscribers, kudos).

The way I found these guides to be useful was when I was actually playing the game, for example in the Witcher 3 while collecting the griffin school gear, I constantly switched back and forth and alt+tabbed out so I would be able to find all the pieces. 

Research took time though and even if we disregard the actual search process or having to read multiple sites when looking up these pieces of information, I still had to read the actual content even the parts that were completely irrelevant to my current situation in the game.

And you guessed it right, in the meantime the game was on pause, running in the background. Then when I found the content, I started the process of switching back and forth between the game and the guide. I’ve been doing this for years and with so many games I’ve lost count.

With Fridai and GPT-3 however, I knew we could find the right way of recreating these guides in a way so it would give me filtered through content, only relevant to me, without having to leave the game – or even using my keyboard or mouse.

The marriage of these two products, would result in a human-like conversation that is tailored to the actual problem I would be facing in the game. Is this the answer to question number 2 and 3 above? Well, partially.

GPT-3 is a super creative technology and it has already been trained for usage, but putting it into an actual product requires a lot more than just connecting to it. In a way, you need to tame its creativity to make sure it works with your product. Therefore the first action we needed to take is to figure out an architecture that would serve the right content to GPT-3, so when it parses the unstructured content, it would be able to digest and produce the right piece of advice.

Then we needed to figure out where to get started, in terms of which game we should go after first. A game’s knowledge base can be very complicated, with hundreds of items, monsters, characters, quests and the list goes on. In order to make the experience work, we would need to dissect the game into the smallest parts, so whenever a user requires guidance about a monster or alchemy, we could provide the best possible response. Therefore we needed a game that we all knew, had little upcoming changes to its knowledge base but also had users who could then test it for us. This is why we picked the Witcher 3: Wild Hunt.

After really taking the game apart, in terms of content we identified the moving parts of how many elements exist in the game of which we will need to provide some form of content to the engine.

We had:

  • 132 monsters
  • 521 characters
  • 250 alchemy items
  • 304 pieces of armor
  • 30 witcher contracts

And more…

We collected individual pieces of content of all the elements of the knowledge base we have identified – either by having gamers in our community write something or looking up an actual guide and reformatting it. Then we saved all these pieces, made them searchable and tagged them.

Afterwards we needed to make sure, when the user’s query comes in about a given situation, we find the right piece of content, parse it and generate a response with GPT-3, format it to fit the game experience and feed it to the text-to-speech engine. Thus providing a real time, filtered piece of information about the game to the end-user.

This made us start working on an NLP (Natural Language Processing) layer to understand the user’s query, connect it to elasticsearch to find the correct file and then feed it to GPT-3, where we could take the most advantage of its creativity to craft the best response after parsing the unstructured data. (Note: I will make a post about the architecture of Fridai in a separate post).

And you can see the results in this video below, step by step:

This is a video of Mark playing the Witcher 3 with Fridai

At the end, user experience speaks for itself. GPT-3 works its magic on content that we can feed to it, however if there are no filters or NLP layers preceding the engine, it would just start crafting stories or irrelevant content for the user’s query.

The question of building a scalable, efficient way to keep Fridai’s knowledge base have been solved by the elastic engine that constantly updates itself from online guides and community generated content, while GPT-3 allows users to work with different queries, therefore getting different responses for “List the weaknesses of the griffin” or “Tell me some tips about defeating the griffin” – just the way you would ask your friend about how he solved certain problems while playing.

By the time I am writing this, we will have released Fridai’s version with full Witcher 3 support and I cannot wait to share more of our experiences in how people use this different, more interactive format of a guide or walkthrough to play with the Witcher.

AI Powered Gaming Assistant Gaming Assistant Video games

Fridai, The New Game Play Counselor

Back in the 1980s, Nintendo Entertainment System was on the uphill climb to success and recognized the need to have people readily available to help players get through difficult points in their video titles. The people they employed were known as “Nintendo Game Play Counselors” and, if you think about it, was every gamer’s dream job.

Unfortunately, the advent of the internet was right around the corner, which meant information was readily available at everyone’s fingertips as soon as they started installing dial-up in their home. Once the internet was more accessible in almost everyone’s household by 2005, Nintendo Game Play Counselor’s position was disbanded.

However, it was nice knowing we had people we could rely on in a pinch. Imagine playing the old Nintendo system and getting stuck on trying to beat Mario’s bosses. I ran into this issue with the NES and even with the Bowser fight on the Super Nintendo. Do you know how hard it was without the internet? Nintendo game play wasn’t challenging for a lot of children, but for me, it would have been helpful knowing there was someone I could talk to when I needed help.

As technology started advancing and Nintendo had the capability to put out more challenging games, there was the need for these professional Nintendo Game Play Counselors. Imagine playing thousands of hours worth of games all day so you could answer simple questions about bosses, hidden items, and other issues gamers were having at the time with their Nintendo game play.

Unfortunately, when Nintendo Game Play Counselors were phased out, we were forced to purchase strategy guides or strain our eyes even further on poorly developed websites with black backgrounds and green texts to read walkthroughs. Having that personal touch of hearing another person’s voice, was an experience we were deprived of for so many years. Sure there was YouTube and “Let’s Play” videos, but it never felt personal. No one wants to sit through ten minutes of someone talking about their day as they’re playing when all they want to do is figure out how to get past the part they’re stuck on.

Fortunately, the concept of Nintendo Game Play Counselors is still alive. No, you don’t have to pick up a landline phone to call into someone sitting at a call center ready to help. No, this concept lives on through easily accessible and responsive artificial intelligence known as Fridai, the gamer’s assistant.

How Can Fridai Help Me?

Imagine Fridai as a Nintendo Game Play Counselor, but without having to stop what you’re doing just so you can pick up your phone. Having to stop and minimize a game takes away from the immersion, and it’s just not fun having to sift through every website’s ads just to look for one piece of information.

Fridai can give you animal locations & prices so you can fully immerse yourself into hunting in Red Dead Redemption 2

For example, in Red Dead Redemption 2, there is a lot of information for the game’s content that isn’t readily available in the game. When I encounter a bear, I know it takes the fun away from the game when I have to pause it just to look down at my phone and look up what weapons I need to use to kill it. All I have to say is, “Hey Fridai!” and a comforting “bling” sound assures me that he is live and ready.. “How do I kill a grizzly bear?” I ask it as I don’t have to take my hands off of my keyboard or controller. Fridai then tells me the type of weapon I need to switch to and the ammo I should use for an effective kill.

Now imagine trying to do this with a Nintendo Game Play Counselor. You would have still needed to put your controller down, go to your landline phone, and call their hotline. Imagine Fridai as a similar concept except with more effectiveness and the cool-factor of getting a response as soon as you say something.

You don’t have to switch windows, fumble with your mouse cursor, or accidentally close a game. Fridai is right there, listening to your every need. If you need to step away from your PC for a moment, just say, “Friday, set my Steam status to busy,” and Fridai will set you up. You can also ask Fridai to play your favorite Spotify tunes for you, so you don’t have to stop to do it yourself.

Are You Ready To Give Fridai A Try?

  1. Visit or search for Hello Fridai through your Microsoft Store 
  2. Download & sign up
  3. After logging in, browse through Fridai’s menu to learn more about what it offers.
  4. Enjoy your gameplay without ever having to leave it.