Text tools

What changed: Diff

Two versions of a text, a note before and after someone edited it, look alike but differ somewhere. The Diff library finds where. DiffCompare takes the old text, the new text and how many unchanged lines of context to show around each change, and gives back a diff: a line starting with - was taken out, one starting with + was put in, and one starting with a space is context, unchanged:

put "apples" & return & "bread" & return & "milk" & return into tOld
put "apples" & return & "butter" & return & "milk" & return & "tea" & return into tNew
put DiffCompare(tOld, tNew, 1) into tChange
put line 1 to -1 of tChange

@@ -1,3 +1,4 @@
 apples
-bread
+butter
 milk
+tea

This is the unified diff format that programmers' tools such as git use, so a diff made here can be read there. The @@ line says where the change sits: lines 1 to 3 of the old text became lines 1 to 4 of the new one. Each line here ends with a return, as the lines of a text file do; a last line without one is told apart with a line \ No newline at end of file, so it shows as changed when a line is added after it. The diff's own lines end with a return too, and line 1 to -1 of leaves off the last one, so the Message Box shows no empty line after it.

A diff is also a recipe. DiffPatch applies one to the old text and gives the new text, so a program can keep the first version and the small diffs after it, rather than every version whole:

put "apples" & return & "bread" & return & "milk" into tOld
put "apples" & return & "butter" & return & "milk" & return & "tea" into tNew
put DiffCompare(tOld, tNew, 1) into tChange
put DiffPatch(tOld, tChange) is tNew
put DiffCompare(tOld, tOld, 1) is empty

true
true

Two texts that are the same give an empty diff, which is a quick way to ask "did anything change?". DiffCompareFiles does the same as DiffCompare with two file paths, reading the files for you.

Because every changed line starts with its sign, a script can count the changes, or show them in colour:

put "one" & return & "two" & return & "three" & return & "four" into tOld
put "one" & return & "2" & return & "three" & return & "4" & return & "five" into tNew
put 0 into tAdded
put 0 into tRemoved
repeat for each line tLine in DiffCompare(tOld, tNew, 0)
   if tLine begins with "@@" or tLine begins with "---" or tLine begins with "+++" then next repeat
   if char 1 of tLine is "+" then add 1 to tAdded
   if char 1 of tLine is "-" then add 1 to tRemoved
end repeat
put tAdded && "added," && tRemoved && "removed"

3 added, 2 removed

A changed line counts as one taken out and one put in, which is how diffs see the world: two did not become 2, it was removed and 2 was added in its place.

A command line's options: GetOpt

A program run from a terminal is given its settings as options after its name: report -v --output=summary.txt notes.txt. Short options are a dash and a letter (-v), long ones two dashes and a word (--verbose), and what is left, here notes.txt, are the arguments. GetOpt sorts them out. Its first parameter, the grammar, lists the options it should know, separated by spaces, each with its other names before it after commas; a = at the end means that option takes a value:

local tCommandLine
put "-v" into tCommandLine[1]
put "--output=summary.txt" into tCommandLine[2]
put "notes.txt" into tCommandLine[3]

put GetOpt("v,verbose o,output=", tCommandLine) into tParsed
put tParsed["options"] into tOptions
put "verbose" is among the keys of tOptions
put tOptions["output"]
put tParsed["arguments"][1]
put the number of elements in tParsed["errors"]

true
summary.txt
notes.txt
0

The answer is an array of three: options, keyed by each option's last name in the grammar, however it was written; arguments, numbered; and errors. -o summary.txt, -osummary.txt and --output summary.txt all mean the same, and short options can share one dash: -hv is -h -v.

A real program leaves out the second parameter, and GetOpt reads the commandArguments, the words the program was started with. Anything it cannot make sense of goes into errors, a message each, so a program can say what is wrong and stop:

local tCommandLine
put "-x" into tCommandLine[1]
put "--output" into tCommandLine[2]

put GetOpt("v,verbose o,output=", tCommandLine) into tParsed
put the number of elements in tParsed["errors"]
put tParsed["errors"][1]
put tParsed["errors"][2]

2
Unknown option '-x'
Missing argument for 'output' option

File types and email: MIME

The MIME library, met in Using extensions, knows the MIME type of each kind of file, the label a web server or an email gives it. It also works the other way, from a type to the file endings that have it:

put mimeTheMIMETypeOfExtension("jpg")
put mimeTheMIMETypeOfExtension("JSON")
put mimeTheExtensionsOfMIMEType("image/jpeg")

image/jpeg
application/json
jpe
jpeg
jpg

An email can carry only plain letters, digits and a few signs, so anything else is encoded first. mimeEncodeForMIMETransfer does both of email's encodings: quoted-printable, which leaves plain letters alone and writes the rest as = and two hex digits, and base64, which turns any bytes into letters. Text is encoded as UTF-8 first:

put mimeEncodeForMIMETransfer("Café au lait", "quoted-printable")
put mimeEncodeForMIMETransfer("Café au lait", "base64")

Caf=C3=A9 au lait
Q2Fmw6kgYXUgbGFpdA==

mimeEncodeAsMIMEEmail builds a whole email, headers, body and attachments, and puts it in it, ready to hand to a mail server. Each attachment is an element of an array, with its name and its data (or a filepath), and its type comes from its name:

local tAttachments
put "notes.txt" into tAttachments[1]["name"]
put "Bring the map." into tAttachments[1]["data"]
put mimeEncodeForMIMETransfer("See you at noon.", "quoted-printable") into tBody
mimeEncodeAsMIMEEmail tBody, "ada@example.com", "grace@example.com", empty, "Lunch", tAttachments
put it into tEmail

repeat for each line tLine in tEmail
   if tLine begins with "From:" or tLine begins with "To:" or tLine begins with "Subject:" then put tLine
end repeat
put "notes.txt" is in tEmail
put "text/plain" is in tEmail

From: ada@example.com
To: grace@example.com
Subject: Lunch
true
true

Sending it is another job, for a mail server and its password; the IDE's tsNet library can do it, as the Dictionary's page for mimeEncodeAsMIMEEmail shows. Never put a mail password in a stack you give away.

function changedLines pOld, pNew
   local tAdded, tLine
   repeat for each line tLine in DiffCompare(pOld, pNew, 0)
      if tLine begins with "+++" then next repeat
      if char 1 of tLine is "+" then put char 2 to -1 of tLine & return after tAdded
   end repeat
   return line 1 to -1 of tAdded
end changedLines

put changedLines("red" & return & "green", "red" & return & "blue" & return & "gold")

blue
gold