Detachable terminal sessions

You can use ''screen'' or ''dtach'' to make sure that if your connection is dropped from a linux server, that your session is preserved and you can link back to it. This is especially useful for keeping processes running and being able to come back and check on them later.

Detachable sessions with SCREEN

Screen is very powerful, it's like a text-terminal window manager. You can create a session with multiple terminal windows all detachable and reattachable in one screen session.

Steps:

  1. Make sure you have ''screen'' installed on the target server. From a session on that server, run sudo yum install screen
  2. Start a ''screen'' session by entering >screen -S ''mysession'' at the prompt (replacing ''mysession'' with a name you'll remember when you re-attach)
  3. screen is mostly run with ctl-a <something> key combos, for example, ''ctl-a h'' will bring up a help screen, and ctl-a ctl-c creates a new shell.
  4. To re-attach a session, log back in to the machine, and then you can re-attach with screen -r ''mysession''

Detachable sessions with dtach

Another, less powerful but lower-overhead tool to use that lets you detach like in screen but doesn't do the other screen stuff is dtach. It is not as powerful as screen, but has the advantage that you can see with the ps command what is running what socket.

Steps:

  1. Make sure you have dtach installed on the target server. From a session on that server, run sudo yum install dtach
  2. dtach uses named sockets, so you will be creating a socket in /usr/tmp, and running a bash session (type dtach --help or check the ''man'' page for details):
  3. in the bash session, run whatever program you want
  4. Detach your session with ctl-\ key if you want
  5. Re-attach later (even after logging off) with dtach -a /temp/checkbotrun

How to Set Up a User-level Perl Library

If you can't figure out how to get elevated perms to install Perl modules and apps for everyone to use, you can set them to your local user account. Below where it says ".profile" you may need to change depending on your linux/unix environment to .bash_profile. This example also grabs and sets upcpanm on the first line, which you can skip if you already have it: curl http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib` echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' ~/.profile echo 'export MANPATH=$HOME/perl5/man:$MANPATH' ~/.profile

You will now have a local perl library, <tt>~/perl5</tt> where all subsequent perl module installs you do from the command line will go.

How to read the system logs on RHEL Machines

system logs are not in /var/log anymore. App and service logs are, ex: Apache, cron. But if you want to see the system log, you need to run jourtnalctl, probably with sudo, a la sudo jourtnalctl. There are details on how to use this command here (also a man page, of course):

How to see what services are running on an RHEL linux server

Two steps:

How to add a user to a linux server

To add a new user:

ex: <tt>sudo useradd &lt;username&gt; --groups webpub -p &lt;password&gt;</tt>

(the '''-e''' expires the account, if you have to put in an expire date, use YYYY-MM-DD format for expire date. You can expire the account *after* you create it, with <tt>usermod</tt>: ex: <tt>usermod &lt;username&gt; -e </tt> )

The password has certain rules (can't be a dictionary word or close), and this is not enforced with useradd and if it *is* too simple, the user won't be able to log in even after you create the account. So '''make it complicated OR you can set the password manually to make sure it passes after you create it as above''':

ex: <tt>sudo passwd &lt;username&gt;</tt> New password: BAD PASSWORD: it is too short BAD PASSWORD: is too simple Retype new password: etc

Also, if you want to add them to the sudoers group, an admin has to use sudo visudo to edit the sudo file in vi, add the new user like the other people in the list: ## Read drop-in files from /etc/sudoers.d (the # here does not mean a comment) #includedir /etc/sudoers.d trusteduser ALL=(ALL) ALL ... &lt;username&gt; ALL=(ALL) ALL

How to preview your Markdown document

.md files in git are written in Markdown. Even if you find a tutorial and figure it out, you may not be using a tool with preview to write or edit your markdown document. Here is one: https://daringfireball.net/projects/markdown/dingus

How to pretty-print JSON at the command line

Check out this page: http://www.skorks.com/2013/04/the-best-way-to-pretty-print-json-on-the-command-line/

For quick-and-dirty, if python is already installed, you can use a python tool <tt>mjson.tool</tt>, you just pipe the output to it, ex: curl http://stp-trt-otpapi.soundtransit.org/otp/routers/default/index/agencies | python -mjson.tool |less

There is a C tool called [[http://lloyd.github.io/yajl/ YAJL]] mentioned there that works great that can be grabbed to your machine with yum to a linux virtual:

sudo yum install yajl

You can cat the file and pipe it into the YAJL tool like so, for example, if you wanted to save the file (first example) or don't care to (2nd example): wget --output-document=test.json http://oba-otp-prod.soundtransit.org/otp/routers/default/index/agencies/1 cat test.json | json_reformat | less

OR curl http://oba-otp-prod.soundtransit.org/otp/routers/default/index/agencies/3 | json_reformat | less

The other advantage of YAJL is that its a library and it has bindings that allow it to be used in other languages like perl and ruby

How to add up a column of numbers in a text file in Unix shell

You can use sed or perl to get your file to just a list of numbers, then this will give you a running total in the form of "{current added value}, {current total}": perl -anle '$x+=$_ for(@F); print "$_, $x"; END {print $x}' filename.txt

How to add a prefix to a set of files in Unix shell

There is a bash thing called [[http://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Brace-Expansion Brace Expansion]] that just expands what's in the braces, so for example: for i in *.txt; do mv {,40_}$i; done;

If you had one file in the folder, "test.txt" the above would issue the command mv test.txt 40_test.txt. The comma at the beginning means no prefix, and if you wanted a bigger expansion for different purposes, you can add more \{,40_,50_\} etc

How to pretty-print CSV files at the command line

From https://www.stefaanlippens.net/pretty-csv.html (follow the Mac directions)

Make a script with the special less flags and perl described in that website: in your user ~/bin folder (create it if you haven't).

For example, create a zsh/bash script file called ~/bin/pretty-csv.sh, containing:

#!/bin/bash perl -pe 's/?<=,)|(?<=^,/ ,/g;' "$@" | column -t -s, | exec less -F -S -X -K

Make it executable (chmod ug+x ~/bin/pretty-csv.sh) and create a bash alias for it (e.g in .zshrc or .zprofile)

alias pretty_csv='~/bin/pretty-csv.sh'

Afterwords, you can invoke it on a file: pretty-csv stops.txt

or pipe an output to it: sed -ne '/Line/p' routes.txt | pretty-csv

How to prepend text into stdout output in Unix shell

There are two ways to do this:

  • You can use semi-colon in bash and bash-like shells to, say prepend the columns from a CSV file to a grep. You may need to use paretheses as well.
  • * Example: <tt>(head -1 routes.txt ; grep "[A-Z] Line" routes.txt ) | pretty-csv</tt>

  • You can use a parenthesis with a ''"(prepend text && cat)"'' construct, as follows. I guess this is better if you are building from left to right and remember you need to prepend after you already started typing your pipe command
  • * Example: <tt>grep "[A-Z] Line" routes.txt | (head -1 routes.txt && cat) |pretty-csv</tt>