pyaed - a python audio entropy daemon

Same game again. Not all my boxes have tv-cards to leach entropy from, so I needed some other source. The soundcard comes into mind quickly, and every box has a soundcard nowadays. The existing audio_entropyd once again wasn’t useful, because what it produced didn’t survive the FIPS-140-2 tests (aka wasn’t really random at all on my box). I then went on reimplementing the exact same algorithm it uses in python with pyaudio to take a closer look on the data. When I dumped the output of this algorithm into a file, I could even see patterns in hexdump of that file. strange. Well, there must be some randomness, so I went on implementing a different algorithm. It also records stereo audio and then looks at the upper bit (0×0001) in the samples. If this bit is different on both channels and the current two stereo samples are different from the last two, it records that as an entropy bit (you can argue about that, though - afaik randomsound uses the same mechanism). To add some more confidence in the entropy, it then XORs 64kbit of entropy into a 4kbit block. This way, it’s getting around 3kbit/s of entropy out of the soundcard.

download sourcecode

README:

Python Audio Entropy Daemon v0.0.1 (works on my machine)
(c) 2010 by Kai Dietrich

Inspired by audio_entropyd by Folker Vanheusden
http://www.vanheusden.com/aed/
and randomsound by Daniel Silverstone
http://www.digital-scurf.org/software/randomsound

This software is Licensed under the
GNU General Public License 2.0 or later.

System Requirements:
--------------------
Python 2.6
PyAudio 0.2.3
a soundcard with line or mic in
optional: rng-tools / rngd

What it does:
-------------
Pyaed records samples from an audio input device, extracts some noise/entropy
and writes it to a fifo.

Pyaed opens the default audio input device pyaudio finds and records frames
(44.1kHz, 16bit, stereo). It looks at the highes bit (0x0001) in the samples from each channel.
If these bits differ and the samples are different from the last (to ignore constant signals),
a bit of entropy is recorded. To increase the qualitiy of randomness, it then compresses 64kbit of
entropy into 4kbit by XORing the bits. It then writes the bits into a fifo.
You can then attach rngd from the rnd-tools to this fifo (rngd -f -r entropy.fifo).
rngd will test the noise with a FIPS 140-2 test for it's statistical randomness
and delivers the bits to the kernel entropy buffer.

It does not work, what can I do?
--------------------------------
a) read the code (it's not that much)
b) fix the code
c) Play around with alsamixer to get noise on the default input device,
   turn up boosts and input levels until you get levels around 50%. If you want to, you can even put
   in a stereo mic to get noise from the air and not just the electromagnetic noise from the ADC.

How can I enhance the code?
---------------------------
Just do it. If you like this tool, you can just set up a project somewhere
and start collecting improvements. For me this was just some fire-and-forget
single-task code.

pyved - a python video entropy daemon

Well, what do computer scientists do, when they are bored? They toy around with cryptography.
For some reason I didn’t get video_entropyd to run (it would throw v4l errors and segfault), but I desperately need entropy. Now what I came up with is a quick python script which does essentially the same thing, but with much more dependencies and high-level scripting languages. Also I just grab video frames from the TV-card with pygame.camera, extract the entropy and write it to a fifo. All the communication with the kernel then does rngd from the rng-tools. It picks up the bits, checks if they are really random and only then puts them into the kernel. All in all I have a solution, which (according to rngd) generates about 80MiBit/s of entropy from a good old Bt878 receiver. Im quite satisfied :)

update: it turned out, that was only a number which resulted from reading and writing in chunks. The long run performance is 8kbit/s of entropy.

download source code

Python Video Entropy Daemon v0.0.1 (works on my machine)
(c) 2010 by Kai Dietrich

Inspired by video_entropyd by Folker Vanheusden,
The main part actually is just a python version of Folkers code.
http://www.vanheusden.com/ved/

This software is Licensed under the
GNU General Public License 2.0 or later.

System Requirements:
--------------------
Python 2.6
PyGame 1.9.1
a video4linux device
optional: rng-tools / rngd

What it does:
-------------
Pyved records frames from a video4linux device, extracts the noise/entropy
and writes it to a fifo.

Pyved opens the first video4linux device it finds and records frames (720x576, RGB).
If it finds the kernel entropy pool to be empty it starts extracting noise
from two successive frames. Every uncorrellated change in one of the three color
channels is considered to be a bit of physical randomness and written to
the fifo "entropy.fifo". You can then attach rngd from the rnd-tools to this
fifo (rngd -f -r entropy.fifo). rngd will test the noise with a FIPS 140-2 test for it's
statistical randomness and delivers the bits to the kernel entropy buffer.

How fast is it?
---------------
On a Pinnacle Bt878 analogue TV card, tuned to a really bad channel,
rngd reports the following speeds (entropy bits per second):

stats: HRNG source speed: (min=1.330; avg=1.783; max=4.657)Gibits/s
stats: FIPS tests speed: (min=70.382; avg=88.529; max=89.969)Mibits/s

this is frickin fast, compared to all those commercial devices

It does not work, what can I do?
--------------------------------
a) read the code (it's not that much)
b) fix the code
c) tune your tv-card with a tuner application to some channel before starting pyved

How can I enhance the code?
---------------------------
Just do it. If you like this tool, you can just set up a project somewhere
and start collecting improvements. For me this was just some fire-and-forget
single-task code.

What is Open Source?

Well, here is the final answer. Just created it from the texts in the OpenSource Jahrbuch 2004 to 2008. Took me some hours and some code-snippets from my diploma theses. Just a quick Porter-stemming (which doesn’t work to well with german, I know) and a manually created stopwordlist. There’s definitly room for improvement (better stemming, differential corpus analysis, say agains the gutenberg corpus) but I just need it for putting people in the right mindset.

Wordle: OpenSource Jahrbuch

Wiki2beamer 0.8 is out (update)

Some weeks ago, already, I released wiki2beamer 0.8. It’s mostly a maintenance release. It now works again with python 2.4 which makes it easier to run on ancient systems (like our universities ;) ), has a litle bug fixed where “expressions” immediatly following lists were not transformed (so you had to add a newline) and the license was changed to “GPL 2.0 or later” so we don’t get stuck in some copyright problems as time moves on.

I also put the manpage for wiki2beamer online here (via man2html) so google can find it and non-*nix users can read it, too.

I should probably finish my thesis before I start coding at wiki2beamer, but if anyone out here is interested, there still are some things to be done:

  1. Create a Lessig-style slide environment, like the [code]-environment
  2. Split the code into a commandline wiki2beamer frontend and a python module as backend
  3. Write a formal syntax description, so we can create a real parser/compiler instead of these regular expression tricks.
  4. Build distro packages. (Fedora, RedHat, Arch, … anyone? Gentoo already has one. Update: I’ve built a Debian/Ubuntu package now, too. Go get it at SourceForge.)
  5. Build a windows installable package. (?)
  6. Update online documentation.

Also, we finally seem to get users :)
According to the sourceforge download statistics, we had 86 downloads in the last 2 months.

So, add one and head over to get wiki2beamer 0.8.