Results 1 to 10 of 79

Thread: Just released: create your own stage notes with voice commands

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Crew Chief Mega Corp CEO mr_belowski's Avatar
    Join Date
    Feb 2017
    Posts
    1,815

    Just released: create your own stage notes with voice commands

    This has just been released (15th October 2020) and opens up some interesting possiblities - there's nothing quite so arse-clenching as blasting over a flat out crest with only the pace notes you just created on a single sighting-run between you and certain death.

    For Dirt Rally 2, first 2 minutes is me going the stage recce recording pace notes, then I restart the stage and run it in anger:


    For Richard Burns Rally, this is a fast run after I created the pace notes


    CC allows you to build your own stage / pace notes with voice commands during a stage recce run.

    It works like this:
    - start the stage with the app running in the right mode
    - make the voice command "start stage reccy"
    - when you pass a corner / obstacle, make a clear voice command. This is surprisingly hard, you only get 1 shot at it and the voice command must be delivered clearly and reasonably quickly. "left 3 don't cut keep in" would work, "left 3 don't cut" then a second later "keep in" might not.
    - there needs to be at least half a second or so between voice commands.
    - cruise the stage slowly doing all this - crests, bumps, ruts, fords, bridges, corners etc. You have to concentrate and it's easy to get muddled up. Remember to speak clearly and make each command an obvious single set of instructions. This is the hardest part, it takes practice
    - if the app doesn't hear it'll tell you and you'll need to repeat the instruction so if you're going too fast this will be difficult / impossible to get the note in the right place
    - if you make a call and it's clearly wrong (you screwed it up or it's been mis-recognised, you can replace the last call you recorded by saying "correction, [then the new correct call]"
    - when you complete the stage make the command "finish stage reccy"once you've done that, you'll have a new file CrewChiefV4/RBR/[stage name]/pacenotes.json. You can inspect it or you can just tell the game to restart the stage immediately and see if you survive

    pacenotes.json has each pace note along with the voice command that triggered its creation and any fragments of the voice command that the app wasn't able to process here's an example where I said "caution ford" but the app misheard is as "caution four"
    Code:
    {
        "Distance": 1659.67273,
        "Pacenote": "detail_caution",
        "Modifier": "none",
        "RawVoiceCommand": "caution four",
        "UnprocessedVoiceCommandText": "four"
      }
    so rather than calling out "caution, ford" it just called "caution". Normally that last line is empty, and a single voice command may create more than one pace note - e.g.
    Code:
      {
        "Distance": 1326.0437,
        "Pacenote": "detail_caution",
        "Modifier": "none",
        "RawVoiceCommand": "caution square right don't cut"
      },
      {
        "Distance": 1336.0437,
        "Pacenote": "corner_square_right",
        "Modifier": "detail_dont_cut",
        "RawVoiceCommand": "caution square right don't cut"
      }
    One final note, for rbr the pace notes created with this mechanism will replace any that the game may be sending - if the app sees a pace notes file for the current stage it'll use this instead of the games own notes

  2. #2
    Love the work you are doing! Pacenotes have been the biggest problem for me in this otherwise great game. I also use VR and since we can't use the pacenote plugin inside VR it's a hazzle to get good pacenotes.
    Is it possible to get the CC overlay inside rbr? It is sometimes hard to get the correction, I ussualy get "matching partial response right", "Unable to find a pacenote to correct" so I have to say it a few times. Ussualy I'm sitting in the ditch after a off trying to correct the pacenote.

    Other than that it feels very solid! A list of possible commands would be great also, is it possible to add both long and tightens to a corner?

  3. #3
    Crew Chief Mega Corp CEO mr_belowski's Avatar
    Join Date
    Feb 2017
    Posts
    1,815
    Not sure why the CC overlay wouldn't work for you with RBR in VR - it works fine for me.

    Regarding voice recognition, the corrections stuff is a bit more challenging than the initial pace notes creation.

    The expectation is that it's a short command that only corrects a corner's tightness or its modifiers (tightens / long / cut / don't cut are the main ones). So a typical correction command would be made immediately after you pass (exit) a corner, and it would be simple like "correction, 4" or "correction, don't cut". The tightness (1, 2, 3, 4, 5, 6, flat, hairpin, square, k, medium, fast) can be combined with a modifier in a correction so "correction, 4, cut" should work. The app will apply the correction to the last corner it thinks you drove. If you also include a direction in the correction ("correction, right 3") then it will attempt to apply the correction to the most recently driven corner in the specified direction. If you're getting "unable to find pace note to correct" try making the correction without the direction.

    The possible commands list is quite big. Most rally-type commands are supported. The best source for the list of commands is here in the source code:
    https://gitlab.com/mr_belowski/CrewC...ion_config.txt

    starting at line 450

    it's still got some work left to do on it. For example, if you make a command "logs outside" there's no sound for this, so the app will just say "keep in". Same for "rocks inside" - it'll be called as "keep out".


    these can be combined so you can have "left 3 long tightens then opens don't cut", or "caution flat right over crest cut" and stuff like that with a single command. There is a limit to the number of items a command can have - any more than 5 items in a command and the speech recogniser will start to get really unreliable.

  4. #4
    Sorry I totally missed that there was an option for steamVR overlay. I'll give that a try tonight!

    I tried it on a stage and the notes came out great on the first try, I had some troubles with threes and fours and I had to say them a few times to get it to recognize the note. This is probably because of my speech/accent as English is not my main language.

    Is there a command to remove the previous note during recce? For example I had my "left 5 jump" become "left 4 jump", would it be possible to say "replace, left 5 jump" and it would replace the latest note with the new. This would remove the need for reversing back to get the timing correct also.

    I was very impressed with the timing, how do you deal with that? Is just a flat distance or are you trying to figure out the speed carried through the corners and then figuring out how far in seconds to the next note?

    I have been doing stage rally IRL for 2 years now so if you ever need some help or have any questions feel free, love the work you are doing. It's really great for the rally community and is making RBR much more accessible for more people.

  5. #5
    Crew Chief Mega Corp CEO mr_belowski's Avatar
    Join Date
    Feb 2017
    Posts
    1,815
    3s and 4s can be replaced with "slow" and "medium", the speech recogniser doesn't care whether you use numbers or descriptions for corners and you can mix and match any combination of number, description, and direction first or direction last - the corner names the speech recogniser understands are

    "1" / "acute"
    "2" / "k"
    "3" / "slow"
    "4" / "medium"
    "5" / "fast"
    "6" / "slight"
    "flat"
    "square"

    so "4 left", "left 4", "medium left" and "left medium" should all work the same.

    if you make a pace note and need to correct it whilst doing a recce, you use "correction, left 5 jump". This deletes the most recently added pace note and replaces it with the one after "correction". It'll keep the distance data from the pace note it just deleted so in your example the "left 5 jump" will replace the "left 4 jump" and will be called at the same distance that the original note would have been called, even if you've travelled a bit before you made the correction command. Obviously this isn't perfect and if your recce is driven quickly and you need to make corrections it can get messy.


    The timing uses some simple corner / obstacle length estimates based on the pace note you just created. It assumes that you've just passed the corner / obstacle when you make the recce instruction so it subtracts some distance from your current lap distance, taking into account things like corner speed, 'long' modifier and stuff, then estimates where the obstactle / corner might have started and uses that for the pace note's distance. The app also adds a separate 'distance to next corner / obstacle' placeholder pace note at the actual distance where you made the instruction and (depending on a few things) it might call a distance pace note at using this point.

    We then use the car's speed to decide when to play each pace note using a look-ahead estimate (this is configurable, and can be adjusted on the fly during a stage with the "earlier calls" / "later calls" commands). The app also uses the car's speed and the number of calls it needs to read out to decide whether it should use the relaxed or the rushed versions of the sounds to give a sense of urgency where needed. I like the idea of this feature but I'm not sure it sounds quite 'right' yet.


    Glad it's working well for you. It's a complicated piece of functionality and it's very hard to get right. It works great for me and I'm way faster through a stage using my own pace notes . We're always open to feedback so will welcome any suggestions

  6. #6
    One thing I would like to be added after doing a few stages now is to get feedback if the correction has registered. And maybe have two different recce modes. So currently the recce mode is great, but then what I'll would like to do is run the stage again, slowly and correct all errors while the codriver is reading the notes to me. I could do that by running the stage normally but I get no feedback that the correction has registered. So what I suggest is have another mode where he reads out the correction you just made for when you are driving the stage slowly correcting the pacenotes, and the other mode is when you are running the stage at speed then maybe you only get a "Got it" as acknowledgement as you still want to hear the rest of the notes.

    EDIT: As a workout for now is it possible to get the console text big and clear in the steamOverlay? I can't seem to find a good option for this.
    EDIT2: I manage to get the steamoverlay showing the entire CC app but unfortunately the app console does not scroll by itself to the latest message
    Last edited by Andreas Zetterlund; 31-10-2020 at 17:56.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •