Text processing

Ed

Ed, an interactive, line-oriented text editor, was originally written by Ken Thompson in 1971.
It has two modes, the command mode and the input mode.
Entering the input mode is completed by typing ‘a’ (append), ‘i’ (insert) or ‘c’ (change).
The input mode is terminated by entering a single period (.) on a line.

From Ed forked two lines:

Ex (Extended editor). From Ex developed Vi (Visual editor), which had a visual (2-dimensional) user interface. Ex merged in Vi
Sed (Streaming Editor), a non-interactive, line-based text-editor.

Sed

Sed was developed by Lee E. McMahon in 1973/1974.
sed.sourceforge.net the official site of sed
www.rtfiber.com.tw/~changyj/sed/
www.grymoire.com
A new feature of GNU-sed is in-place editing, which is done with the option -i.
Some usage examples:

Change the date format:
sed 's/\([0-9]\{2\}\)\([0-9]\{2\}\)\([0-9]\{2\}\)/\3\2\1/'

delete empty lines:
sed -i '/^$/d'

Vi

In 1976 vi was written by Bill Joy
In 1987 Stevie, a clone of vi, was written. It had a good implementation of the 'u' (undo) and '.' (repeat) commands.
www.vim.org exists since 1991, based on Stevie and written mostly by Bram Moolenar
ex-vi

Modes

ViM is a modal editor.
The three main modes and how to get there:

Normal mode, which you are in on opening a file.
In this mode You jump around in the text, record and execute macros.
A trick is that [Ctrl]-c can be used instead of [Esc] to get from either insert mode or from command line mode into normal mode
Insert mode.
Here You can insert text.
- i I: at the position of the cursor / at the beginning of the line
- a A: after the position of the cursor / after the end of the line
- o O: in the line following / before the current line
- c C: change
- s: substitute
Command line mode, also called ex-mode.
Type a colon in normal mode and You are at the command line of ViM at the bottom of the screen.

Normal mode

Moving the cursor, navigating in the text
- h, j, k, l: left, down, up, right
- w, e, b: next word, end of word, back one word
- W, E, B: next word, end of word, back one word
- (, ): next paragraph up, down
- {, }: next paragraph up, down
- /, ?: find next/previous occurence
- ^F scroll Forward, ^B scroll Back, ^D scroll Down, ^U scroll Up
- Marks:
  - ma: set mark "a" at the position of the cursor
  - 'a: go to the line where the mark "a" is
  - `a: go to the position of mark "a"
yanking/copying text
- y preceding any cursor movement yanks the text
- yy yanks a line
Deleting/cutting text
- d preceding any cursor movement deletes the text
- x deletes a character
- dd deletes a line
Pasting text:
- P pastes text before the cursor
- p pastes text after the cursor
Recording macros ("complex repeats"):
- qa starts to record macro a
- q quits the recording of the currently recorded macro
- @a executes macro a, in this case this is everything, which was done between qa and q
- 10@a executes macro a 10 times
Folds: ViM's synonym for fold is z, due to the form of the letter "z".
zf creates a fold, zo opens a fold, zc closes a fold

Command-line mode

Substitute: s
Global: g
Marks:
- :marks: list all marks
- :delmarks a: delete mark "a"
- :delmarks!: delete all marks
Search for a newline with \n, but insert a newline with [Ctrl-V][CR]
:sp [Ctrl]-w [Ctrl]-w
= idents area marked as visual block
:help command
justify/align: le, ce, ri
wrap: set textwith (tw), gq visual block, [Esc]!}fmt, set wrap/wrapmargin is concerned with the screen size which is different on different machines,
A shell command is initiated by "!".
:r !date
reads the output of "date" to the current line.
Settings for Hebrew and Arabic:
rl/norl: rightleft/norightleft
ri/nori (revins/norevins): reverse insert/ no reverse insert
keymap=hebrewp: hebrew phonetic
keymap=arabic

Features which are found only in the full installation of Vim
(sometimes only vim-tiny is installed by default)

syntax highlighting: 100+ languages supported
auto word completion ctrl-p
directory browsing
ruler: shows line/row of cursor at the right bottom of the screen

c2.com/cgi/wiki?IfYourCarWereVim
The CPAN module text-vimcolor uses the Vim text editor to highlight text according to its syntax, and turn the highlighting into HTML, XML or PDF output.
.vimrc must be in UTF if it contains utf-characters in abbreviations
vimdiff
http://vanhemert.co.uk/vim/vimacros

Meta-characters/regex

Grep

Grep was created by Ken Thompson in 1973 as a standalone application adapted from the regular expression parser he had written for the ed editor he had created.
The name comes from the ed editor command, g/re/p (global / regular expression / print).

Awk

Awk was developed by Aho, Weinberger and Kernighan, hence the name AWK, in 1977, inspired by Grep. awk.info using multiple lines in awk and sed

delete the line break, join lines:
awk 'BEGIN { FS="\n"; RS=""} { gsub("\n", ""); print}' a

symbol	ed	vi/ex	sed/grep	awk/egrep
search (left-side)
.	x	x	x	x	any char
*	x	x	x	x	none or more
^	x	x	x	x	beginning of line/string
$	x	x	x	x	end of line/string
\	x	x	x	x	mask
[]	x	x	x	x	group
	x	x	x		saves pattern
\1,\2	x	x	x		saved pattern
{}				x	number of
\{\}	x		x		number of
\<\>	x	x			beginning/end of word
+				x	one or more
?				x	none or one
\|				x	or
()				x	groups
substitute (right-side)
\	x	x	x	x	unmasks
\1\2	x	x	x	x	or ()
&	x	x	x	x	search pattern
~	x	x			previous pattern again
%				x	previous pattern
\u\U	x	x			upper case
\l\L	x	x			lower case
\e	x	x			undoes previous \u\l
\E	x	x			undoes previous \U\L

Most Regex which work in Perl work in Vim.
In Vim they require an escape "\"; e.g. "+" in Perl is written as "\+" in vim.
However, "x to y occurences" is written "\{x,y}" in ViM, with a backslash only in the beginning. Other editors:

mined (since mid-80ies; written for Minix-OS; extensive unicode and CJK support)
Joe-editor (since early Linux distributions)