README - Silas' website

Introduction

This is the README page of the site pointed by silas.net.br. If you are seeing this page from this URL, you are seeing the HTML rendered version. But if you are seeing it at its github repository, you are seeing the reStructuredText version.

Basically, this site is built using reStructuredText and converted to HTML using the docutils processing system. Its building system is made on the top of NetBSD make.

In summary, this website follows what is known as semi-static (or semi-dynamic) website. In this concept, the HTTP server deal with plain HTML files, that means that it doesn't deal with code that is dynamically generated. Dynamic sites use languages like PHP to generate HTML at the request moment.

Semi-dynamic websites have their HTML content generated from other files at a specific time and make those files available for the HTTP server, so this server only have to deal with plain HTML files.

The advantages (or disadvantages) or this technique is not the focus here. Anyway, the concept adopted on this site was inspired by the NetBSD website source, which is worth to have a look.

How this website source is organized

On the top of the source (where this file belongs to), there are some directories and subdirectories with RST files. There is one special directory, though, that is share/, that holds all logic about building.

For every (or almost every) directory, there is a Makefile that tell the build system what to make with each file or subdirectory in the current directory. So, this is the Makefile for the notes/unix/ directory at the time of I write this:

RSTDOCS             = dec-tips.rst

SUBDIR               = linux
SUBDIR              += netbsd
SUBDIR              += x11

.include "../../share/mk/web.site.mk"

Here, the SUBDIR variable is a list of subdirectories that should be analyzed. The RSTDOCS variable holds a list of the files that should be compiled from RST to HTML. Finally we include the share/mk/web.site.mk file (the core of the building system), passing its relative path.

Variable that define targets

Possible variables for Makefiles like those are:

RSTDOCS

List of files to be compiled from RST to HTML.

It is important to point that all generated HTML files are post-processed to have some files references fixed. This site is entirelly written in reStructuredText files that will be converted to HTML files. So, how a file should refer to another file of this website? To the original RST file or the resulting HTML file?

The answer is: to the RST file. After calling rst2html, the resulting HTML is post-processed to make some substitutions, like converting all links to RST files to the resulting HTML file.

This is done to prevent just-in-time RST renderers to refer to non-existent HTML files, like GitHub one.

The RSTDOCS also accepts files which extension is .rst.nw, i.e., files that are actually noweb files and should be preprocessed first to generated .rst files. It is important to note that some substitutions can be made.

SUBDIR: List of subdirectories to be analyzed at the same way the current one is being.
COPY: List of files only to be copied to the destination directory, where all HTML files belong to. Those files are not compiled, but just copied.
TMPFILES: Some programs like LaTeX create temporary files when compiling the original file to another. The build system (web.site.mk) cannot track it, so it is necessary to tell it what these files are. This file is done with TMPFILES which holds a list of every file that should be deleted after the current directory has been processed.

CLEANFILES

Files that are created by other means (like a local target) and are not tracked by web.site.mk must be in this variable, so the make clean command can delete them later.

RENAME_RULES

The redirect framework allows one page to get redirect to another page. It is useful mainly for renames when the old link is already spread over the web. One should use RENAME_RULES variable to add rules and RENAME.x.NEW and RENAME.x.OLD variables to add the source and the target of the redirection. For instance:

RENAME_RULES = busybox
RENAME.busybox.NEW = notes/linux/busybox-troubleshooting.html
RENAME.busybox.OLD = doc.notes/unix/linux/busybox-troubleshooting.html

RENAME_RULES += xen
RENAME.xen.NEW = notes/xen/xen-setup-netbsd.html
RENAME.xen.OLD = doc.notes/xen/xen-setup-netbsd.html

In this example, we define two files on the old doc.notes directory should have redirect instructions to the new files on notes directories. See the redirect.mk file inside share/mk directory of this website source for more information.

NOTANGLE_RULES

The notangle framework process .rst.nw files to extract source code embeded in page. It uses the notangle tool from noweb. Like RENAME_RULES, you should use it and its helper variables:

NOTANGLE_RULES = osdev-inc
NOTANGLE.osdev-inc.INPUT = osdev-inc.rst.nw
NOTANGLE.osdev-inc.DESTDIR = osdev-inc-code
NOTANGLE.osdev-inc.TARBALL = osdev-inc-code.tar.gz
NOTANGLE.osdev-inc.CHUNKS += mbr1/mbr1.S
NOTANGLE.osdev-inc.CHUNKS += mbr1/Makefile
NOTANGLE.osdev-inc.CHUNKS += mbr1/run.sh

In this example, it will process osdev-inc.rst.nw files and extract mbr1/mbr1.S, mbr1/Makefile and mbr1/run.sh files inside osdev-inc-code directory. It will also generate a osdev-inc-code.tar.gz tarball. For more information, see the notangle.mk file.

The `share` directory

The share directory is where all the magic is. Its subdirectories are:

css/: where style sheets for this site remain.
html/: Holds the template.html file. This is a bit different from the docutils file because it has some configuration options for MathJaX, the engine I use to display math in HTML.

mk/: This directory holds the web.site.mk file, which is the main file of the building system. It is included by other Makefiles and does the hard work, calling scripts described above to compile file HTML pages. Heavily inspired on the NetBSD website source web.site.mk file. Check Variable that define targets for information on the variables that web.site.mk process. It also includes the redirect.mk file that implements the redirect framework and the notangle.mk that process noweb files to extract source code.

rst/: htmltop.rst holds the top menu that will be added to every HTML page. license.rst has the text with the indication Licensed by CreativeCommons By-Nc-Sa 4.0. Every rst file is obliged to have this text at the bottom of the page. The existence of this text is checked in compile time and an error is triggered by web.site.mk if it is different from license.rst.
utils/: Has general utilities used by this site, like nw2rst.sh, a noweb backend to reStructuredText written by me (for more information, see its code.

Building

First, clone this website from its github repository:

$ git clone git://github.com/silasdb/www.git

If the git protocol is blocked, use HTTP instead:

$ git clone https://github.com/silasdb/www.git

For building, you will need docutils, NetBSD make and noweb. If you run GNU/Linux or other non-NetBSD operating system, there is a portable version called bmake which is where that link will just guide you to.

Since you have NetBSD make and docutils installed -- and both make (or bmake, if you run non-NetBSD) and the rst2html program from docutils are in your $PATH, cd to the www directory and compile it:

$ cd www/
$ make # or bmake, on Non-NetBSD