Various other software may also be found within the CVS repository, although these may merit some highlight.
mmstatd | apache-mmstat | mmftpd | mmmail | mmspawnd2 | mmserver-mmserver2 | mmlist-mmpool-mmhash | pthread_util | js-appserv-sh | ecl-unix | bktr | tap-bridge | utf-8 | xisop | mystic-ships | paradise-adventure | symbol-game | irclog | crow-httpd
mmstatd is a small database server, C library and command-line
client designed to store key-based counters for statistics or simple
configuration storage. These up to date statistics may then be queried in
real-time. The C library allows C software to query and modify those using
transactions, and the server internally uses a recovery log.
mmstatd is used by
apache-mmstat. It may be found at the following
This utility can easily be used with apache so that it pipes a custom log
format trough it, for various statistics to become maintained by
At the time I wrote this FTP server, none that I knew were open source
for unix, properly and easily supported virtual users (simulating chroot
for each), running fully unprivileged and under a real chroot(2).
I also wanted a single file to serve as an accounts database, and another
simple configuration file. Internally using the
library, it initially used GNU Pth threads. Over time, the
pthread_util library was written to solve some missing
functionality from POSIX threads, and the
mmserver library was
modified to use pthread. Later on, IPv6 support was also added.
Other features include configurable per-address rate and connection limits as well as bandwidth throttling.
Following the writing of
mmftpd, I also wanted a
non-privileged, chroot-capable non-relaying SMTP server and its POP3
server companion. Originally, MySQL was used as the backend, but
eventually the code was modified to use PostgreSQL and file storage for
message content, which the setups running it had migrated to.
mmftpd it also uses the
so now uses POSIX threads and supports IPv6 as well.
This mail server is highly tuned against spam. It performs various sanity checking and allows filtering based on headers or user/mailbox specific envelope based filters which users may configure via an HTTP interface. The latest HTTP interface to MMMail is unfortunately not open source, however.
Other features include configurable per-address rate and connection limits as well as bandwidth throttling.
mmspawnd2 consists of an
inetd-like server but
an instance of the server is needed for every service. However, more
control is allowed on a service, such as setrlimit(2), chroot(2), running
unprivileged, binding privileged ports to then drop privileges before
starting serving connections, doing some sanity checking before executing
the target executable, etc. This server uses the
library and thus supports IPv6 and maintains a dynamic pool of pre-forked
processes for performance. Connection and rate limits are also supported.
mmserver and mmserver2(top)
mmserver was initially a library for writing unprivileged
network IPv4 TCP servers using the GNU Pth portable user-threads library.
Although mainly I/O bound, which user-threads can be suited for, a special
facility was designed using a pool of processes along with a messaging
system so that parallel tasks, or ones which might require a fair amount of
stack, be served by non-threaded processes.
As POSIX threads implementations became more widely available and more
mature, conforming to POSIX more and more (LinuxThreads didn't conform and
NetBSD had no kernel threads at the time), the
library was eventually written and
mmserver was modified to
use POSIX threads. The non-threaded pool of processes is still available
and may still be used for cases where threads are unappropriate, like for
using non-threadsafe library calls, tasks needing a lot of stack, etc.
Later on, support for IPv6 was also added. This library is used by
mmserver2 consists of another similar library but which is
designed to use processes rather than threads. This model is more adequate
for some types of applications and provides enhanced reliability. A pool
of processes is maintained to serve the clients in parallel, such as is
done in Apache 1.3, and
mmserver2 supports AF_LOCAL, IPv4 and
IPv6 TCP/Stream and UDP/Datagram.
mmserver2 is used by
js-appserv, and has also been used
in proprietary server code. This library also permits to reuse, and to
eventually recycle processes of the pool to for instance recover memory
leaks from a third party library, and this is tunable by every application.
Connection/request rate and limits are also supported.
mmlist, mmpool, mmhash(top)
mmlist is a library of C macros to help dealing with doubly
linked lists, which are a common data structure used in software.
mmpool consists of an efficient fixed-sized objet pool-based
allocator, with some slab-allocator features such as asynchroneous object
creation/destruction and reuse of recently used memory pages.
mmhash is a general purpose hash table library, also a
frequently used data structure. Custom 32-bit hash functions, as well as
object-specific comparision functions may be provided, and the tables may
be tuned/customized via various options. Collisions are dealt with,
as well as automatic table resizing and rehashing.
pthread_util library provides utility features for use
with POSIX threads. It first implements an efficient Amiga/PTH like message
passing interface for inter-thread synchronization, where rings can be used
for polling. It then also implements a poll(2) variant which can be used
to poll on both file descriptors and message rings at a time. Along with
this new polling primitive, asynchroneous connect(2)/accept(2) variants are
implemented. Other facilities include a managed pool of threads to which
tasks may easily be dispatched in parallel. Finally, some thread sleep
variants are also implemented which may also wake up upon events on a ring.
This library was initially written to replace missing POSIX threads library
features which the GNU Pth library allowed, to then allow migrating
mmserver (and thus,
mmmail), to POSIX threads. It also has found other uses
js-appserv and js-sh(top)
The shell (
js-sh) allows to run various programs written in
js-appserv), it consists of a versatile server supporting
IPv4, IPv6 and local sockets. This server is written in C but delegates
Various libraries are provided such as the following:
- Syslog - Access to the syslog(3) logging facility
- Errno - Mappings for C errno
- FD - File Descriptor related syscalls, including file, network sockets, non-blocking I/O and polling
- Event - libevent for OS-specific efficient events polling
- GD - Full GD graphics library support for live image generation/manipulation
- PG - Full Postgresql 8.1.4+ database server (a production quality, featureful and efficient free ORDBMS) support, including asynchroneous non-blocking functions, pre-processed queries and escaping-avoiding explicit parameter-aware functions
- Dir - Directory traversal opendir(3) and friends wrapper
- FS - Various file-system related syscalls
- File - Full stdio FILE * support including popen(3) functionality
- jsdoc - An application to translate comments from JS and SpiderMonkey based C code modules, into reference documentation (currently HTML, but other modules could be written such as for mdoc macros). This implementation was especially written to not care about code, but only about recognized comments.
- Reference documentation of the classes, their properties and methods.
Eventual support might be provided for:
- Signal - Changing signal masks and providing signal handler functions/events
- Global - Support for inter-process shared memory
- MMStat - Support for mmstat(3) statistics database system
Also provided are example programs:
- irclog - An IRC logging robot
- jsthumb - An image gallery indexer application using GD
Here are the relevant links:
- the js-sh and js-appserv source and examples
- the reference documentation for the implemented classes, their properties and methods
- example js-sh applications
ecl-unix is an interface to NetBSD syscalls and various other
important libc interfaces for the ECL implementation of Common Lisp. It is
written to use ECL's native C inline features. This allows ECL to be used
on NetBSD to develop various kinds of server and client applications
easily, with good performance and minimal dependencies. Support for
kqueue(2) is also included.
It makes uses of custom SETF expansions and lexically scoped MACROLETs to override DEFMACRO-defined global macros to very efficiently inline C structure accessors where the code is intended to be compiled. In cases where serializing whole structures at once is more convenient, this is used instead of working via structure accessors. For interpreted or third party code, where random access to a C structure object is necessary, accessor functions are provided for C structures.
This code is still a work in progress.
bktr_multicast, bktr_client and bktr_httpd(top)
bktr_multicast consists of an UDP multicasting server,
listening to BSD
bktr(4) interfaces (device for bt8x8 video
capture devices). It marks images with timestamp and source, then
compresses them in mjpeg format. It then broadcasts those frames over the
LAN, after tagging them with the current timestamp, source and a type of
noonce (to prevent replay attacks) and an RSA signature (to prevent third
party frame insertion, allowing clients to assert their authenticity).
Capture rate and parameters are configurable, as well as the simplistic
motion detection system. Was used on NetBSD with a PCI 4 bktr ports board.
bktr_client is an SDL-based client which displays frames sent
by bktr_multicast for the wanted source. It supports pause and save
features. It also ensures to only decode frames with a valid RSA
signature. This client was used successfully on NetBSD, Linux and Windows.
bktr_httpd is an efficient, I/O bound single-threaded HTTP
server using kqueue(2), which can stream image frames originating from
bktr_multicast source to many web clients.
It supports HTTP authentication and Keep-Alive, as well as some type of
bandwidth modulation to adapt the frame rate to the link speed of every
client. Only frames with a valid RSA signature are streamed.
This is a modular daemon designed to act as a bridge/proxy between a physical ethernet device and a NetBSD tap(4) virtual one to learn more about network protocols and permit to easily write prototype network stacks and low-level applications as userspace modules. Packet capture and injection are possible, and a few facilities are provided to modules for ethernet/IP header deserializing/serializing as well as delayed send queues and a facility for persistent property/variable which may last when modules are reloaded (i.e. at reception of SIGHUP). A diagram of the system can be found here.
More and more applications make use of the UTF-8 external format and of unicode, and UTF-8 is also the standard format of several network protocols. This library implements a UTF-8 to UTF-32 decoder and an UTF-32 to UTF-8 encoder. I needed a comprehensive and efficient C interface which was under a non-restrictive license, for use as third party extensions to software which could not be under the GPL, and had to write this implementation.
Once upon a time, the author was an Amiga user. This was a microkernel with intefaces and design similar to that of AmigaOS. An eventual offspring of Xisop had to be licensed under more restrictive terms and development stopped on the open source original, but here is the old code anyway. This version only runs on Amiga (it was developped on NetBSD using an m68k cross-compiler targetted at the UAE emulator at the time).
The Mystic Ships(top)
After discovering Netrek, playing it, running a bronco server for some time, and hosting a third party CVS repository for bronco-server and cow-client, I got interested in possibly developing a similar game from scratch. There also was an interleving research project which could have benefited from it.
Some work was done on the basic system including an encrypted and compressed, adaptively streamed TCP-based client-server protocol, a kqueue(2) based server allowing a large number of concurrent clients efficiently and an SDL-based portable client (NetBSD, Linux, OSX, Windows). However, as the challenging parts which were interesting to me were fully implemented, interest in the game itself diminished and development was stopped. However, a little effort was put in 2014 to revive it, so that it is at least playable.
- The TMS home page, description, instructions and precompiled clients (updated 2014)
- The Mystic Ships portable client and BSD server source
Since a few of my family members have a drawing, painting and music hobby, and that I have a type of nostalgy for old adventure games and am myself musician and compositor, I started writing an SDL-based text+image adventure game. It started out well, but lack of time and lack of family support stalled development. The existing part is more a demo than anything else, and is portable (tested on NetBSD, Linux and Windows). An early screenshot is also available.
If I ever pick the project up seriously again, I'll most probably use ECL with it such that the world and events system be implemented in Common Lisp.
Please note that although the code is BSD-licensed, the artwork (images and music) is not licensed for public distribution or use.
This was a test I wrote when learing Common Lisp. The main goals were to
learn how to develop an application interactively, how to make custom use
of the package system, dynamic symbols manipulation and simple macros.
symbol-game runs on ECL or SBCL.
I previously had written
irclog.js, but wanted to rewrite it
in Common Lisp. This implementation is also more complete as it can
It was nice to develop as a fair amount of progress could be made
interactively in SLIME while not having to reconnect the client to the IRCd.
It was designed for ECL. It will be used to log certain important events
happening on IRC.
Because of the power of Lisp, neat tricks are possible like interactive development, compile-time template to text/html generation, embedding those HTML templates elegantly into code (not as text, but as hierarchical s-expression objects with conditional and loop commands), providing an HTTP-LET for dynamic application user code to easily bind GET/POST/Cookie variables, etc.
It thus consists of an interesting learning experience. If I'm satisfied with it once It's more complete, future actual applications might get written with it. Crow-httpd also currently hosts this site, replacing the previous js-httpd that hosted it for years.