Command line tips

Detachable terminal sessions

Detachable sessions with SCREEN

Steps:

1. Make sure you have ''screen'' installed on the target server. From a session on that server, run sudo yum install screen
2. Start a ''screen'' session by entering >screen -S ''mysession'' at the prompt (replacing ''mysession'' with a name you'll remember when you re-attach)
3. screen is mostly run with ctl-a <something> key combos, for example, ''ctl-a h'' will bring up a help screen, and ctl-a ctl-c creates a new shell.
• A trick to remember is that in screen your normal vertical scrolling won't work. See the help to see how to activate scroll-mod
• Another trick is that ctl-a is co-opted by ''screen'' while you are in a screen session. To get an actual ctl-a, type ctl-a a
  
To detach a screen session so you can logout, type ctl-a d inside of screen, then you can log off (if you are disconnect, the detach happens automatically)
4. To re-attach a session, log back in to the machine, and then you can re-attach with screen -r ''mysession''
• If you log in to a machine, you can check that there's a screen session with screen -list, and connect to a detached session with screen -r

Detachable sessions with dtach

Steps:

1. Make sure you have dtach installed on the target server. From a session on that server, run sudo yum install dtach
2. dtach uses named sockets, so you will be creating a socket in /usr/tmp, and running a bash session (type dtach --help or check the ''man'' page for details):
• dtach -c /tmp/checkbotrun bash creates a process run by dtach with the socket /tmp/checkbotrun
3. in the bash session, run whatever program you want
4. Detach your session with ctl-\ key if you want
5. Re-attach later (even after logging off) with dtach -a /temp/checkbotrun
• when you log back into the machine after detaching and logging, off, the job will appear as the initial dtach command, e.g. dtach -c /tmp/checkbotrun bash

How to Set Up a User-level Perl Library

  curl http://cpanmin.us | perl - -l ~/perl5 App::cpanminus local::lib
  eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`
  echo 'eval `perl -I ~/perl5/lib/perl5 -Mlocal::lib`' >> ~/.profile
  echo 'export MANPATH=$HOME/perl5/man:$MANPATH' >> ~/.profile

You will now have a local perl library, ~/perl5 where all subsequent perl module installs you do from the command line will go.

How to add a user to a linux server

ex: sudo useradd <username> --groups webpub -p <password>

(the '''-e''' expires the account, if you have to put in an expire date, use YYYY-MM-DD format for expire date. You can expire the account *after* you create it, with usermod: ex: usermod <username> -e )

The password has certain rules (can't be a dictionary word or close), and this is not enforced with useradd and if it *is* too simple, the user won't be able to log in even after you create the account. So '''make it complicated OR you can set the password manually to make sure it passes after you create it as above''':

ex: sudo passwd <username>

 New password: 
 BAD PASSWORD: it is too short
 BAD PASSWORD: is too simple
 Retype new password: 
 etc

Also, if you want to add them to the sudoers group, an admin has to use sudo visudo to edit the sudo file in vi, add the new user like the other people in the list:

 ## Read drop-in files from /etc/sudoers.d (the # here does not mean a comment)
 #includedir /etc/sudoers.d
 trusteduser         ALL=(ALL)       ALL
 ...
 <username>          ALL=(ALL)       ALL

How to preview your Markdown document

Original Daring Fireball markdown site

How to pretty-print JSON at the command line

The best way, according to a page on skorks.com

"The best way"


• For quick-and-dirty, if python is already installed, you can use a python tool mjson.tool, you just pipe the output to it, ex:

curl http://stp-trt-otpapi.soundtransit.org/otp/routers/default/index/agencies | python -mjson.tool |less

• There is a C tool called YAJL mentioned in that article that works great that can be grabbed to your machine with yum to a linux virtual:
sudo yum install yajl

YAJL project on github


You can cat the file and pipe it into the YAJL tool like so, for example, if you wanted to save the file (first example) or don't care to (2nd example):

wget --output-document=test.json http://oba-otp-prod.soundtransit.org/otp/routers/default/index/agencies/1
 cat test.json | json_reformat | less

OR

curl http://oba-otp-prod.soundtransit.org/otp/routers/default/index/agencies/3 | json_reformat | less

The other advantage of YAJL is that its a library and it has bindings that allow it to be used in other languages like perl and ruby

How to add up a column of numbers in a text file in Unix shell

You can use sed or perl to get your file to just a list of numbers, then this will give you a running total in the form of "{current added value}, {current total}":

perl -anle '$x+=$_ for(@F); print "$_, $x"; END {print "=====\nTotal " . $x}' filename.txt

For example, you could use this with the SDF 'dues' command to calculate your total SDF dues:

$ dues | cut -c46-51 | sed -e '1,2d' |perl -anle '$x+=$_ for(@F); print "$_, $x"; END {print "=====\nTotal " . $x}'


36.00 , 36
18.00 , 54
60.00 , 114
18.00 , 132
18.00 , 150
=====
Total 150

How to add a prefix to a set of files in Unix shell

There is a bash thing called "Brace Expansion" (link below) that just expands what's in the braces

So for example:

for i in *.txt; do mv {,40_}$i; done;

If you had one file in the folder, "test.txt" the above would issue the command mv test.txt 40_test.txt. The comma at the beginning means no prefix, and if you wanted a bigger expansion for different purposes, you can add more \{,40_,50_\} etc

Brace Expansion man page


How to pretty-print CSV files at the command line

Make a script with the special less flags and perl described in that website: in your user ~/bin folder (create it if you haven't).

For example, create a zsh/bash script file called ~/bin/pretty-csv.sh, containing:

#!/bin/bash
perl -pe 's/((?<=,)|(?<=^)),/ ,/g;' "$@" | column -t -s, | exec less  -F -S -X -K

Make it executable (chmod ug+x ~/bin/pretty-csv.sh) and create a bash alias for it (e.g in .profile or .bashrc, or on macintosh: .zshrc or .zprofile)

alias pretty_csv='~/bin/pretty-csv.sh'

Afterwords, you can invoke it on a file:

pretty-csv stops.txt

or pipe an output to it:

sed -ne '/Line/p' routes.txt | pretty-csv

Pretty-csv


How to prepend text into stdout output in Unix shell

There are two ways to do this:
• You can use semi-colon in bash and bash-like shells to, say prepend the columns from a CSV file to a grep. You may need to use parentheses as well.
• Example: (head -1 routes.txt ; grep "[A-Z] Line" routes.txt ) | pretty-csv
• You can use a parenthesis with a ''"(prepend text && cat)"'' construct, as follows. I guess this is better if you are building from left to right and remember you need to prepend after you already started typing your pipe command
• Example: grep "[A-Z] Line" routes.txt | (head -1 routes.txt && cat) |pretty-csv