How to set aliases

First of all, to actually set an alias you have 2 options.

1. $ git config --global alias.co checkout

   The command line way of setting aliases is pretty straightforward. In this case co is an alias for checkout.

2. Editing .gitconfig

   Your .gitconfig might look like this:

   [alias]
   co = checkout
   

   You can also use "!" to execute the aliased command in a shell.

   di=!git status --porcelain --short --ignored | sed -n '"s/^!! //p"'
   

Inspiration for aliases

What are the most used commands in your case? Commit? Checkout? Then set them as aliases. In most cases that's the best way of improving your workflow.

If you want something "future proof" there are a couple of great resources for that.

• Prezto's Git aliases

  Probably the best organization of aliases. They are actually supposed to be used from the shell, not as a git subcommand, but more on that later.

• Git Aliases of the Gods! - Git Merge 2017

  This is a talk by a BitBucket developer Tim Pettersen. Some of his aliases didn't make into my list for complexity reasons, but if you are interested in serving your local Git repository to the internet, you might want to check out the video at 11:32.

• Your imagination

  You need something more? Add it yourself!

Warnings

While creating your aliases you must consider multiple things:

1. Git aliases are case insensitive

   git gco # Lowercase
   git gCo # is actually the same
   

2. Creating way too many shell aliases will probably result in command name clashes.

   For example gs, which I use for git status will clash with GhostScript's executable. Fortunately for me I pretty much never use GhostScript, so that's a loss I can live with.

   You can also source an executable by typing its whole path to skip aliases, like /usr/bin/gs.

Actual aliases

First of all, we are going to "translate" Prezto's aliases for the shell to their respective Git versions.

During the conversion we will lose git submodule and git flow aliases since they are using uppercase letters to differentiate from git stash and git fetch.

[alias]
  # Branch (b)
  b=branch
  ba=branch --all --verbose --verbose
  bl=branch --verbose --verbose
  bc=checkout -b
  bx=branch --delete
  bm=branch --move
  bs=show-branch

  # Commit (c)
  c=commit
  ca=commit --all
  cm=commit --message
  cam=commit --all --message
  co=checkout
  cop=checkout --patch
  cf=commit --amend --reuse-message HEAD
  chp=cherry-pick
  cr=revert
  cres=reset "HEAD^"
  cs=show

  # Data (d)
  d=ls-files
  dc=ls-files --cached
  dd=ls-files --deleted
  dx=ls-files --deleted
  dm=ls-files --modified
  do=ls-files --other --exclude-standard
  dk=ls-files --killed
  di=!git status --porcelain --short --ignored | sed -n '"s/^!! //p"'

  # Fetch (f)
  f=fetch
  fa=fetch --all
  fc=clone
  fp=pull
  fr=pull --rebase
  fpr=pull --rebase

  # Grep (g)
  g=grep
  gi=grep --ignore-case
  gl=grep --files-with-matches
  gL=grep --files-without-matches
  gv=grep --invert-match
  gw=grep --word-regexp

  # Index (i)
  ia=add
  iap=add --patch
  iu=add --update
  id=diff --no-ext-diff --cached
  idw=diff --no-ext-diff --cached --word-diff
  ir=reset
  irp=reset --patch
  ix=rm -r --cached

  # Merge (m)
  m=merge
  mnc=merge --no-commit
  mf=merge --ff
  mnf=merge --no-ff
  ma=merge --abort
  mt=mergetool

  # Push (p)
  p=push
  pf=push --force-with-lease
  pa=push --all
  pt=push --tags
  pat=push --all && git push --tags
  pc=!git push --set-upstream origin "$(git-branch-current 2> /dev/null)"
  pp=!git pull origin "$(git-branch-current 2> /dev/null)" && git push origin "$(git-branch-current 2> /dev/null)"

  # Rebase (r)
  r=rebase
  ra=rebase --abort
  rc=rebase --continue
  ri=rebase --interactive
  rs=rebase --skip

  # Stash (s)
  stsh = stash --keep-index
  staash = stash --include-untracked
  staaash = stash --all
  s=stash
  sa=stash apply
  sx=stash drop
  sl=stash list
  sd=stash show --patch --stat
  sp=stash pop
  ss=stash save
  ssu=stash save --include-untracked

  # Working Copy (w)
  ws=status
  wd=diff --no-ext-diff
  wdw=diff --no-ext-diff --word-diff
  wrs=reset --soft
  wrh=reset --hard
  wc=clean -n
  wcf=clean -f
  wcfd=clean -df
  wx=rm -r
  wxf=rm -rf

Then, we are going to add pretty looking logs based on this StackOverflow answer.

  # Log (l)
  l1 = log --graph --abbrev-commit --decorate --date=relative --format=format:'%C(bold blue)%h%C(reset) - %C(bold green)(%ar)%C(reset) %C(white)%s%C(reset) %C(dim white)- %an%C(reset)%C(bold yellow)%d%C(reset)' --all
  l2 = log --graph --abbrev-commit --decorate --format=format:'%C(bold blue)%h%C(reset) - %C(bold cyan)%aD%C(reset) %C(bold green)(%ar)%C(reset)%C(bold yellow)%d%C(reset)%n''          %C(white)%s%C(reset) %C(dim white)- %an%C(reset)' --all
  l = !git l1

Would be cool to save a snapshot of changes without committing changes, right?

  snapshot = !git stash save "snapshot: $(date)" && git stash apply "stash@{0}"
  ls-snapshots = !git stash list --grep snapshot

Listing recent branches is sometimes useful.

  recent-branches = !git for-each-ref --count=15 --sort=-committerdate refs/heads/ --format='%(refname:short)'

In the end we'll add URL shorthands to popular websites.

[url "https://aur.archlinux.org/"]
  insteadOf = "aur:"
[url "ssh+git://aur4.archlinux.org/"]
  pushInsteadOf = "aur:"
[url "https://github.com/"]
  insteadOf = "gh:"
[url "git@github.com:"]
  pushInsteadOf = "gh:"

General tips

These aren't really aliases but they are still useful.

• Verbose commit messages by default (includes commented out diff)

  [commit]
      verbose = true
  

• Default push location

  [push]
      default = upstream
  

• Enable rerere

  [rerere]
      enabled = true
  

• Enable colored output

  [color]
      ui = true
  

But in the end...

Those aliases might be an overkill for your use case. If you ever feel like you are wasting more time on "upgrading" your workflow instead of getting actual work done, consider stopping.

That being said, you can check out the full .gitconfig in my dotfiles repo.

Configuring Tmux in a right way for your needs is really important if you want to have a distraction free workflow.

Better Tmux prefix

Prefix key is going to be the most used one. Setting it right is crucial. For me Meta-z combination works really well since both keys are easy to press and I don't use any other keybinds that can interfere with it.

# Better prefix
set -g prefix M-z
unbind C-b
bind M-z send-prefix

At some point you'll also face a situation when you need to use the prefix key in nested sessions, so let's handle that too by setting Meta-a as a keybind.

# Nested Tmux keybinds
bind-key -n M-a send-prefix

Pane navigation

Use Meta+arrow keys without prefix key to switch panes.

bind -n M-Left select-pane -L
bind -n M-Right select-pane -R
bind -n M-Up select-pane -U
bind -n M-Down select-pane -D

Window navigation

Use shift+arrow keys to switch windows.

bind -n S-Left  previous-window
bind -n S-Right next-window

Vim user specific keybinds

You can skip this part if you don't plan on using Vi/Vim or their keybinds. Otherwise comments are pretty self-explainatory.

# Instant vim-mode change
set -s escape-time 0

# Enable modifier keys in vim
set-option -g xterm-keys on

# Use vi keybinds
setw -g mode-keys vi
set -g status-keys vi

Mouse support

Generally you shouldn't be using your mouse if you want to be extra productive but there are exceptions, so we'll set it up just in case:

# Mouse support
set -g mouse on
bind -n WheelUpPane   select-pane -t= \; copy-mode -e \; send-keys -M
bind -n WheelDownPane select-pane -t= \;                 send-keys -M

Copy mode keybinds

This snippet creates a vi-like experience in copy mode.

# Copy mode settings
unbind p
bind p paste-buffer
bind -Tcopy-mode v send -X begin-selection
bind -Tcopy-mode y send -X copy-selection

Increase the history size

# Bigger history
set -g history-limit 10000

Change pane/window counting index

If you prefer keeping 0-index pane/window for special purposes, this might come in handy.

# Start counting windows/panes from 1
set -g base-index 1
setw -g pane-base-index 1

Turn on the aggressive resize

When using the same session on multiple screens, you'll be limited by the smallest screen resolution on all screens. This can be mostly fixed by aggressive resize.

setw -g aggressive-resize on

Use a plugin manager

TPM is a really great way of managing Tmux plugins.

I'm personally using the following ones:

• Tmux Resurrect to restore the session after rebooting
• Tmux Copycat for regex based searches in copymode
• Tmux net-speed to monitor network usage

##Plugins
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-resurrect'
set -g @plugin 'tmux-plugins/tmux-copycat'
set -g @plugin 'tmux-plugins/tmux-net-speed'

##Plugin location
set-environment -g TMUX_PLUGIN_MANAGER_PATH '~/.dotfiles/tmux/plugins/'
##Init
run '~/.dotfiles/tmux/plugins/tpm/tpm'

Themes

I'm using a custom Powerline-like Tmux theme I've made myself.

You can read more about that in Creating a native Powerline theme for Tmux.

Is it worth investing time into this?

If you are using SSH to manage multiple computers/servers, or would like to emulate Terminator like features in normal terminals, then yes.

Tmux also can be a good terminal workspace manager.

My full Tmux configuration is in my dotfiles repo.

In this article I'm going to talk about theming Tmux to look like Powerline.

For those of you interested in the end result here is the theme file.

Source it in your tmux.conf.

Inspired by Tmux themepack.

Determining required features

We are mostly interested in:

• Indication of the prefix activation
• Powerline like window list
• Network usage indication
• Time
• Number of tmux clients
• Hostname
• Username

Tmux theming 101

Update interval for the statusline is being set with set -g status-interval SECONDS.

Colors in Tmux config, are set using Xterm color codes. You can use this cheatsheet to get the right color.

set -g status-fg colour240
set -g status-bg colour233

To theme the left status bar we are going to use variable like client_prefix and #S. Full list is available in man tmux.

Note the usage of Powerline characters. As long as your terminal font supports them, you can just copy and paste them into your configuration file.

To set colors, use #[bg=COLORCODE,fg=COLORCODE] before the characters you want to recolor.

set -g status-left-bg colour233
set -g status-left-fg colour243
set -g status-left-length 40
set -g status-left "#{?client_prefix,#[fg=colour255]#[bg=colour31]#[bold] #S #[fg=colour31]#[bg=colour233],#[fg=colour232]#[bg=colour255]#[bold] #S #[fg=colour255]#[bg=colour233]}"

Same thing for the right side of the status bar.

set -g status-right-bg colour233
set -g status-right-fg colour243
set -g status-right-length 100
set -g status-right "#{net_speed} #[fg=colour236,bg=colour233]#[fg=colour255,bg=colour236] %H:%M #[fg=colour233,bg=colour236]#[fg=colour255,bg=colour233,bold] #{session_attached} #[fg=colour255,bg=colour233]#[fg=colour233,bg=colour255]  #(whoami)@#[fg=colour232,bg=colour255,bold]#H "

Window status (middle part) contains the list of your active windows. Set the display format using set -g window-status-format " #I#F  #W ".

set -g window-status-format "  #I#F  #W  "
set -g window-status-current-format "#[fg=colour233,bg=colour31]#[fg=colour255,bg=colour31] #I#F  #W #[fg=colour31,bg=colour233,nobold]"

And for the currently active window:

set -g window-status-current-bg colour39
set -g window-status-current-fg colour255

Result

Full theme file.

Feel free to check out my Tmux tips and tricks!

• Test changes? Terminal.
• Launch server? Terminal.
• Commit changes? Terminal.
• Docker? Terminal.
• Anything else? Probably terminal...
• Using VIM? Always terminal.

Now imagine the core tool of your workflow looking like this:

Doesn't seem very pleasant, does it?

It's all about the eye candy

Communities like /r/unixporn (it's SFW for the record) aim to solve that issue, by providing eye candy for our trusty terminals:

Looks much better, right? But we can't stop now...

We need more statuses!

NOTE: Screenshots are from Powerlevel9K's config showoff.

Overall a much better experience, except for all the overhead you get from running git status and nvm version every single time you press ENTER.

Slow terminal response issues are really common, even outside of Powerlevel9K.

It's not even about relatively fancy nvm/rbenv prompt segments. VCS segments add a lot of overhead in big repos.

This is how it looks like opening a Linux Kernel repo with most of its history deleted (only 14 commits):

Granted, filesystem cache makes subsequent prompt redraws much faster, but there is still a 200-300ms delay that makes typing really annoying.

Async will help us

Here is the list of async prompts I managed to find (feel free to add yours in the comments):

• Pure
• Statusline
• Powerlevel9K's WIP async branch

NOTE: I'm going to use Powerlevel9K in this example purely because of aesthetics.

Instead of this in antigernrc (you can use whatever ZSH package manager you prefer):

POWERLEVEL9K_INSTALLATION_PATH=$ANTIGEN_BUNDLES/bhilburn/powerlevel9k
antigen theme bhilburn/powerlevel9k powerlevel9k

We are going to use the WIP branch with zsh-async:

#Async library
antigen bundle mafredri/zsh-async

#Theme
POWERLEVEL9K_INSTALLATION_PATH=$ANTIGEN_BUNDLES/dritter/powerlevel9k-async_all_the_segments/powerlevel9k.zsh-theme
antigen theme dritter/powerlevel9k powerlevel9k --branch=async_all_the_segments

Much better results!

As you might have noticed, VCS info isn't blocking us from typing into the terminal and executing commands. That's a huge usability improvement.

You can find my full zsh configuration here.

Happy coding!

Demo

Why Terminator? It seems to be the most consistently detected hotword.
Why Mozart? It's not copyrighted.

Code

You can find the native module code in Vagr9K/native-voice-command-detector.
The actual discord bot that utilizes this is located in Vagr9K/VoiceBot.

General implementation plan

First of all, let's break down the idea into smaller pieces.

There are effectively two main parts of our application:

• The main bot code that will be responsible for:
  • handling the Discord API
  • monitoring user actions for commands
  • responding to commands
• The voice command detection code

This might initially seem strange since the voice command detection code is technically a submodule of the main code, but due to its complexity, it's worth it to split the problem into these pieces.

Implementing the main code

The main code gets easily handled by a NodeJS based application.

Libraries like discord.js and ytdl provide easy to use tooling to solve most of our problems.
Add TypeScript on top to get static typing, compile time checks and very convenient autocomplete based on the definitions of the aforementioned modules and you're set.

The npm ecosystem greatly decreases the development time while also lowering the entry bar for people with small programming experience, making it perfect for the audience of Discord.

From the performance perspective these tasks aren't CPU intensive and mostly wait for the network, meaning that the NodeJS event loop can handle them effectively without any extra optimizations being needed.

Implementing the voice command detection

The naive implementation would consist of us piping the entire audio stream into a speech to text service and parsing the text.
Due to privacy concerns and pricing of these services that's not ethically/economically practical.

It's reasonable to expect that voice commands will be less than 1% of the audio stream data, making the so called "hotword detection" a huge improvement in both regards.
After we've detected the hotword we only need to send a chunk of audio data for speech to text and later command extraction.

Keep in mind that the data that we will be receiving is a stream, meaning that after a hotword is detected we will have to redirect the next X seconds of data into a buffer that will be later sent to a speech recognition service.

It's also worth noting that if a user pronounces another hotword while the previous one was being buffered, we need to create a new buffer for the new command and submit the old one for processing.

The next issue is that the hotword detector is buffering the data on its end too and we'll have to submit the leftover data after the hotword detection to the command speech buffer to make sure that no command speech segments are lost.

This gives us the general logic of the application (data flow is from top to bottom):

Picking the correct libraries

Let's start filling the blanks.

For Hotword detection Porcupine seems to be the best bet.
This is due to:

• Relatively permissive license
• Multitude of available ports
• High performance
• Hotword models can be generated on the fly (but can't be used commercially)

Ideally, Precise would've been used, but it's Python only making things complicated, since it can't be as easily ported to NodeJS.

It's really hard to compare the speech to text services, but judging by the pricing and general feedback the GCloud Speech To Text is a safe choice.

Details, details, even more details

Here things get extra complicated.

First of all, we receive the audio data from Discord as RAW Opus frames. Opus is a highly optimized audio codec that manages to provide high quality audio even at the lowest bitrates, making it optimal for network based voice communication.

Since the supported format for Porcupine is the signed 16 bit low endian PCM (single channel, 16K frequency), we need to first decode the data.
GCloud in its own turn supports S16LE, but due to network payload size concerns, it makes sense to first compress the data via encoding. As we've established before Opus provides the best size/quality and is supported as OggOpus on GCloud. The encoded audio data will also need to be Base64 encoded and sent as a JSON payload by a POST request.

The updated diagram looks like this:

Designing the correct architecture

Since the logical segment is done, we need to dive into the architecture.

• Option #1: Just write NodeJS code.

  This won't scale reliably due to the fact that encoding/decoding/hotword detection are CPU intensive tasks. Once we start blocking the event loop the performance of NodeJS will degrade very quickly. We won't be able to support multiple people speaking in real-time and massive delays will be ensured. Combine this with the fact that NodeJS is single threaded and the processing load can't be distributed between multiple logical CPU cores either.

  Unfortunately, in their current state the worker threads are limited due to object serialization/deserialization restrictions. Namely when it comes to passing functions to worker threads.

• Option #2: Rely on native encoding/decoding libraries.

  Although the encoding/decoding gets sped up massively by using native modules like node-opus, we face the issue of N-API -> JS and JS -> N-API function call and argument copying overhead. This will mostly become a problem with the huge buffers. A couple seconds of speech is around 500+ 16bit Opus frames which decode into 160K+ 16 bit PCM frames (~312KB of data). Combine it with each frame having to be submitted by invoking a native function (extra overhead for each call) things don't look as pretty. On top of that we're still limited to a single thread.

• Option N3: Submit all data to a native module that will process the rest.

  This way we don't deal with the overhead and have a full control over how and in which thread our code executes. Upon finishing the processing we will invoke a callback with the full text to keep the N-API overhead at a minimum.

Designing the parallelization

First of all, let's break down our app to concurrent tasks to see which actions can be parallelized (notice the green rectangles):

This gives us a good idea over which tasks can be assigned to worker threads. The issue is that we're dealing with streams. Processing the buffers only when a new frame enters the module is problematic since a user may pronounce his command and stay silent for the rest of the command's time limit, effectively hanging the system.

This means that our logic processing needs to be handled on periodic intervals by a separate lightweight thread, while worker threads execute the CPU intensive code.

This brings us to the final design:

The VoiceProcessor instance holds all the buffers and their states. Each audio stream (Discord user) gets his own instance.

The main NodeJS thread which originally invokes the native code is only used for submitting the Opus frames to the VoiceProcessor instance buffer. No processing is done during the submission which keeps the event loop blocked for the minimal amount of time.

The periodic timer thread periodically calls the Sync function on each VoiceProcessor instance to perform the state processing on them. This ensures consistent latency irrelevant of the Opus frame submission rate. The main NodeJS thread is also unloaded and doesn't have to perform these actions.

Based on the timers and buffer states of the VoiceProcessor Opus decoding, hotword detection and speech recognition are being invoked in separate worker threads to ensure maximum possible parallelization of work on all the available logical CPU cores.

Benchmark

The benchmark was executed as follows:

• Average command text callback delay is measured
• Prerecorded data is submitted
• It's submitted instantly and assigned different IDs each time to simulate multiple concurrent users
• Silence TTL is 500ms, buffer TTL is 200ms, thus making the lowest possible callback time ~900ms
• GCloud network requests are skipped due to inconsistency they introduce
• Concurrent user count configurations are 1/10/50/100/150/200/250/300/350/400/450/500
• Worker thread configurations are 2th/4th/8th/16th
• The benchmark was run on a Ryzen 1800x (8c/16th, stock, 3200Mhz CL14 RAM)

Results:

Visualized (callback response time is pictured in relation to concurrent users, lower is better):

As expected, 32 worker threads on a 16 logical core system perform the same as 16 workers.

There is a linear scaling with the worker count from 2 to 4, and from 4 to 8 workers.

Jumping from 8 workers to 16 provides ~23% performance improvement which is in line with the general expectations from AMD's SMT implementation.

Overall, the nature of the task allowed proper parallelization to be implemented. This is close to a best case scenario when it comes to multithreading.

It's also worth noting that Discord enforces sharding after your bot connects to 2500 servers. From my observations it's highly unlikely that more than 100 people will be concurrently talking on those 2500 servers. As you can see from the <100 concurrent user tests, we have enough performance to run each shard's instance on a dual core CPU without any issues (ignoring NodeJS's own processing needs).

Conclusion

Voice command detection even in its simplest form is far from being easy. There are a lot of edge cases to consider and performance metrics to keep track of.

As of right now, this is still an alpha stage project, but it shows that using N-API to handle the performance critical segments while keeping the simplicity of NodeJS for the rest of the project, effectively combines the best of both worlds.

This has been my "weekend project" for quite a while. If you think I've missed something, feel free to point it out.