hazardous

vim, tmux, and termguicolors

if you're here to simply grab the code, the complete solution is at the bottom of the page! otherwise, the following is something of a narrated journey of how i came to my solution.

recently i have been shifting to the use of tmux within my remote dev environments as a way to check against data loss in the event of a local crash or disconnect when working via ssh in the terminal.

unfortunately a side-effect was that i was limited in my choice of themes for vim - to avoid hassle, i used themes that did not rely on termguicolors to achieve the attempted look. although i have dabbled in vim theming myself, i still have very little idea how vim themes work - i am limited to taking an already-written colortheme file and replacing the color codes and text styling with my own. maybe one day i'll have learned enough to write my own from the ground-up, and when/if i do, i will definitely try to include a way to adjust on-the-fly for handling tmux detection.

the issue is that when set termguicolors is set in your vimrc, and vim is opened within tmux, you will break your colors and be left with the appearance of a vanilla vim session.

the problem

it dawned on me that there might be a solution, and so i googled around and found this code:

if has('+termguicolors')
  let &t_8f = "\<Esc>[38;2;%lu;%lu;%lum"
  let &t_8b = "\<Esc>[48;2;%lu;%lu;%lum"
  set termguicolors
endif

it worked! and i was content with it for awhile. after i was satisfied with it playing nicely with tmux, i rolled it into my official dotfiles vimrc, so it became standard within any vim setup i would have from then on.

...unfortunately i had been so excited by it working within tmux that i apparently never stopped to see how it worked outside of tmux. some time went by, and i found myself adding some other options and plugins, and so when i discovered i was having issues with mouse support, i didn't realize at first what the culprit could be.

the issue was that if i clicked on an open vim session in a terminal, a bunch of ANSI codes would spit out on the bottom line of VIM. scrolling would also cause a bunch of codes to spit out - but not actually scroll. to fix it i would have to delete the characters, until vim seemed to 'recapture' my mouse and resume the usual way it handles when set mouse=a is enabled.

the solution

my first reaction to noticing this was to of course roll the most recent changes to my vimrc back - but no dice there. i was on the verge of the sort of annoyance that comes with the realization that i might have to 'start from scratch' and test my vimrc by adding one thing at a time to see where it was breaking.

but thanks to whatever random odds, i happened to notice that the random codes that were being spit out when i clicked or scrolled in vim would generally appear as e.g. +6543;44;33M - it seemed to resemble the pattern in the above code, let &t_8f = "\<Esc>[38;2;%lu;%lu;%lum".

it dawned on me that while the above code was allowing me to use set termguicolors within tmux, it was also setting this tmux-specific string outside tmux as well, which was breaking my vim when using it in a normal terminal session.

the first step towards a working solution was realizing if has('+termguicolors') was not the best way to detect tmux - it wasn't really a way to detect it at all. so i replaced it with if exists('$TMUX') - which worked! i no longer had my mouse issues with vim outside of a tmux session... but i no longer had termguicolors set outside of tmux either.

you're probably way ahead of me here, but yes, i immediately realized this would call for an else if type conditional - i'm only starting to get comfortable with this sort of scripting within bash, but i figured that surely, if vim allowed for this sort of powerful option-setting in vimrc, it would also have some sort of else if equivalent.

the code

thus, 

if exists('$TMUX')
  let &t_8f = "\<Esc>[38;2;%lu;%lu;%lum"
  let &t_8b = "\<Esc>[48;2;%lu;%lu;%lum"
  set termguicolors
else
  set termguicolors
endif

boom! a fairly robust, set-and-forget method to ensure termguicolors is set within and outside of tmux, that allows mouse support and 24-bit colors.