LessonsExtensions: Built inLesson 68

Sharing files with TorrentXT

Share files among many computers at once with BitTorrent, through TorrentXT: the one session, its settings, making a torrent, seeding, magnet links, the events a session reports, the files in a torrent, and stopping.

  • Extensions
  • Lesson 68 of 83
  • About 13 minutes
  • 8 examples

Everyone shares

BitTorrent shares a file among many computers at once. Each one, a peer, fetches pieces from the others and gives its own pieces back, so no single server carries it all, and the more people want a file, the faster it spreads. The xTalk Suite's TorrentXT brings libtorrent, the engine inside many BitTorrent programs, to xTalk; its handlers begin with bt.

The examples stay on this computer: they share tiny files of their own, in a folder of their own, and turn off the ways a session finds peers on the internet.

The session

btStartSession starts the one session that runs everything; only one may run at a time, and a second call gives 0. btStopSession stops it, and you must always stop it when you are done: nothing does it for you. The examples keep the session in a global (Scope), so a run that stops halfway with an error is cleaned up by the next:

ExampleRun it in the playground
global gLessonTorrents
if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
put gLessonTorrents > 0
put btStartSession() && "for a second session"
put btListenPort(gLessonTorrents) > 0
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
true
0 for a second session
true

A session has hundreds of settings, each with a name. btSetBool turns one on or off, btSetInt sets a number and btSetString some text; each gives 0 when it works and -3 for a name it does not know, and btGetSetting reads one back. Turning off the DHT (how peers find each other on the internet) and local peer discovery keeps a session to itself:

ExampleRun it in the playground
global gLessonTorrents
if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
put btSetBool(gLessonTorrents, "enable_dht", false)
put btSetBool(gLessonTorrents, "enable_lsd", false)
put btSetInt(gLessonTorrents, "upload_rate_limit", 51200)
put btFormatBytes(btGetSetting(gLessonTorrents, "upload_rate_limit")) && "a second, at most"
put btSetInt(gLessonTorrents, "speed", 1)
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
0
0
0
50 KiB a second, at most
-3

Making a torrent

A torrent describes what is shared: the names and sizes of the files, and a fingerprint of each piece, so every piece can be checked as it arrives. btCreateTorrent makes one from a file or a folder and gives its bytes, to save as a .torrent file and give to others. It needs no session. Its other parameters are the piece size (0 lets it choose), options (0 for the usual) and trackers, servers that introduce peers:

ExampleRun it in the playground
put specialFolderPath("temporary") & "/oxt-lesson-torrents" into tFolder
create folder tFolder
put "Hello, swarm!" into URL ("file:" & tFolder & "/hello.txt")
put btCreateTorrent(tFolder & "/hello.txt", 0, 0, empty) into tTorrent
put the number of bytes in tTorrent > 100
put tTorrent contains "hello.txt"
put tTorrent into URL ("binfile:" & tFolder & "/hello.torrent")
put there is a file (tFolder & "/hello.torrent")
Output
true
true
true

Seeding

To share a file, add its torrent to the session with btAddTorrentFile, saving into the folder that already holds the file. The session checks every piece, finds that it has them all, and starts seeding: giving pieces to whoever asks. btTorrentStatus gives an array about a torrent: its "state" (which btStateName says in words), its "progress" from 0 to 1, its "name", its "totalSize" and more:

ExampleRun it in the playground
global gLessonTorrents
if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
get btSetBool(gLessonTorrents, "enable_dht", false)
get btSetBool(gLessonTorrents, "enable_lsd", false)
put specialFolderPath("temporary") & "/oxt-lesson-torrents" into tFolder
create folder tFolder
put "Hello, swarm!" into URL ("file:" & tFolder & "/hello.txt")
put btCreateTorrent(tFolder & "/hello.txt", 0, 0, empty) into tTorrent
put btAddTorrentFile(gLessonTorrents, tTorrent, tFolder) into tHandle
repeat 100 times
   put btTorrentStatus(tHandle) into tStatus
   if tStatus["isSeeding"] is 1 then exit repeat
   wait 20 milliseconds
end repeat
put tStatus["name"]
put btStateName(tStatus["state"])
put 100 * tStatus["progress"] & "% done," && btFormatBytes(tStatus["totalSize"])
put the length of btInfoHash(tHandle) && "hex digits of info hash"
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
hello.txt
seeding
100% done, 13 B
40 hex digits of info hash

The info hash names a torrent everywhere: two people with the same hash are sharing the same files.

Magnet links

A magnet link is a torrent in one line of text: its info hash and a name. Anyone with the link can fetch the rest from peers. btAddMagnet adds one, with the folder to save into. Here the state stays "fetching metadata", because with the DHT off and no trackers no peer can send the torrent; online, one would:

ExampleRun it in the playground
global gLessonTorrents
if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
get btSetBool(gLessonTorrents, "enable_dht", false)
get btSetBool(gLessonTorrents, "enable_lsd", false)
put "magnet:?xt=urn:btih:0123456789abcdef0123456789abcdef01234567&dn=demo" into tLink
put btAddMagnet(gLessonTorrents, tLink, specialFolderPath("temporary") & "/oxt-lesson-torrents") into tHandle
wait 200 milliseconds
put btTorrentStatus(tHandle) into tStatus
put tStatus["name"]
put btInfoHash(tHandle)
put btStateName(tStatus["state"])
get btPause(tHandle)
wait 100 milliseconds
put btTorrentStatus(tHandle) into tStatus
put "paused:" && (tStatus["isPaused"] is 1)
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
demo
0123456789abcdef0123456789abcdef01234567
fetching metadata
paused: true

btPause stops a torrent and btResume starts it again; btSetTorrentLimits limits its speed, and the queue handlers (btQueueUp, btQueueTop and the rest) decide which goes first.

Events

The session works on threads of its own and keeps a queue of what happened: a torrent added, resumed, finished, paused, an error. btPoll gives every event since the last poll, oldest first, as an array, each with a "name" and the "torrent" it is about. In an app, poll on a timer, or let btStartPolling send each event to your stack as a message with the event's name:

ExampleRun it in the playground
global gLessonTorrents
if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
get btSetBool(gLessonTorrents, "enable_dht", false)
get btSetBool(gLessonTorrents, "enable_lsd", false)
put specialFolderPath("temporary") & "/oxt-lesson-torrents" into tFolder
create folder tFolder
put "Hello, swarm!" into URL ("file:" & tFolder & "/hello.txt")
put btCreateTorrent(tFolder & "/hello.txt", 0, 0, empty) into tTorrent
put btAddTorrentFile(gLessonTorrents, tTorrent, tFolder) into tHandle
repeat 50 times
   repeat for each element tEvent in btPoll(gLessonTorrents)
      if tEvent["torrent"] is tHandle and tEvent["name"] is not "stateChanged" then
         put tEvent["name"]
      end if
   end repeat
   wait 20 milliseconds
end repeat
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
torrentAdded
torrentResumed
torrentFinished

The files in a torrent, and stopping

A torrent of a folder holds every file in it. btFileList lists them, each with its "path", "size", "progress" and "priority", and btSetFilePriority sets one, counting files from 0: 0 skips the file, 4 is usual and 7 comes first. (A path with .pad in it is a padding file, empty space that lines files up with pieces, and nothing is saved for it.) btRemoveTorrent takes a torrent out of the session, deleting its files with true and keeping them with false:

ExampleRun it in the playground
global gLessonTorrents
if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
get btSetBool(gLessonTorrents, "enable_dht", false)
get btSetBool(gLessonTorrents, "enable_lsd", false)
put specialFolderPath("temporary") & "/oxt-lesson-torrents" into tFolder
create folder tFolder
create folder tFolder & "/pics"
put "aaaa" into URL ("file:" & tFolder & "/pics/a.txt")
put "bb" into URL ("file:" & tFolder & "/pics/b.txt")
put btAddTorrentFile(gLessonTorrents, btCreateTorrent(tFolder & "/pics", 0, 0, empty), tFolder) into tHandle
get btSetFilePriority(tHandle, 0, 0)
wait 300 milliseconds
repeat for each element tFile in btFileList(tHandle)
   if tFile["path"] contains ".pad" then next repeat
   replace backslash with slash in tFile["path"]
   put tFile["path"] && tFile["size"] && "bytes, priority" && tFile["priority"]
end repeat
put btTorrentCount(gLessonTorrents) && "torrent"
get btRemoveTorrent(gLessonTorrents, tHandle, true)
wait 300 milliseconds
put btTorrentCount(gLessonTorrents) && "torrents"
put there is a file (tFolder & "/pics/a.txt")
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
pics/a.txt 4 bytes, priority 0
pics/b.txt 2 bytes, priority 4
1 torrent
0 torrents
false

Try it yourself

Write a function magnetFor that takes a file, makes its torrent, adds it to a session to read its info hash, and returns a magnet link with the file's name, ready to send to a friend.

Show a solution
ExampleRun it in the playground
global gLessonTorrents

function magnetFor pFile
   local tFolder, tName, tHandle
   set the itemDelimiter to slash
   put item 1 to -2 of pFile into tFolder
   put item -1 of pFile into tName
   put btAddTorrentFile(gLessonTorrents, btCreateTorrent(pFile, 0, 0, empty), tFolder) into tHandle
   return "magnet:?xt=urn:btih:" & btInfoHash(tHandle) & "&dn=" & urlEncode(tName)
end magnetFor

if gLessonTorrents is an integer then btStopSession gLessonTorrents
put btStartSession() into gLessonTorrents
get btSetBool(gLessonTorrents, "enable_dht", false)
get btSetBool(gLessonTorrents, "enable_lsd", false)
put specialFolderPath("temporary") & "/oxt-lesson-torrents" into tFolder
create folder tFolder
put "My holiday notes" into URL ("file:" & tFolder & "/notes.txt")
put magnetFor(tFolder & "/notes.txt") into tLink
put char 1 to 20 of tLink
put the length of tLink
put tLink ends with "&dn=notes.txt"
btStopSession gLessonTorrents
put empty into gLessonTorrents
Output
magnet:?xt=urn:btih:
73
true

Words in this lesson

Each one finds every lesson that uses it. The Dictionary has the whole story of each, with its syntax and more examples.