LFS: The dark side of power. Part 2

  • Tutorial

Foreword


So, in the previous article, we started collecting LFS, focusing on the fact that we assembled a temporary system that has everything necessary for further assembly of tools.

Now we will build the main system, making the necessary settings for the work along the way. As this article continues the LFS series, without further ado, let's get down to business.

However, before moving on, let's move a little away from the classical scheme proposed by the authors of the book and do this

$ su - lfs
$ wget http://roy.marples.name/downloads/dhcpcd/dhcpcd-6.7.1.tar.bz2 --directory-prefix=$LFS/sources
$ wget http://www.linuxfromscratch.org/blfs/downloads/7.7/blfs-bootscripts-20150304.tar.bz2 --directory-prefix=$LFS/sources
$ logout


The fact is that the standard documentation regarding the assembly of the base system does not describe the network setup process in the case when you will receive an IP address from a DNS provider, or in case a dynamic ip gives you your home router. Or in the event that you are building under a VM that has access to the network through NAT.

Therefore, after we collect everything and everything, we will additionally install and configure a DHCP client that will allow a fresh system to get ip immediately after a reboot and still have access to the network. This step applies already to the BLFS book.

1. Chroot, the Great and Terrible


First of all, make root the owner of the $ LFS / tools directory again

$ su - root
# export LFS=/mnt/lfs
# chown -R root:root $LFS/tools


Before that, it belonged to the lfs user, but this user is on the host system and it is not there (there are no users yet!) On the system we are collecting. All further actions will be performed from root in the section where the system is going. In this regard, we need to solve the following problems
  • Problem # 1 (main): All the paths that we will use should start from the root of the real system. This is vital for the normal operation of the entire environment and the kernel of the system.
  • Problem # 2 (concomitant): taking into account the fact that the installation paths of binary files in the assembled system and the host system coincide (or almost coincide), it is necessary to "fence off" the system directories of the host and the assembled system from each other.

Both problems are solved by the chroot command, which allows changing the root directory of the system on the go. The chroot execution field on the $ LFS partition all paths to LFS directories will begin with "/", that is, we create the appearance of work in the root of the system being built. Those directories that are above $ LFS (that is, the / mnt / lfs directory) will be completely cut off and there will be no access to them. This is great - all the actions that we perform as root will not affect the host system. But we will need access to VFS - the virtual file system of the host machine, formed by the directories / dev, / proc, / sys and / run. We remember one of the basic concepts of unix - "everything is a file." We need access to equipment and system resources, and it is done through work with VFS objects.

Access to the host system directories from chroot is possible if you first mount the necessary directories to the corresponding LFS directories. To do this, perform the following operations. Create VFS Mount Points

# mkdir -pv $LFS/{dev,proc,sys,run}


Create character devices / dev / console and / dev / null

# mknod -m 600 $LFS/dev/console c 5 1
# mknod -m 666 $LFS/dev/null c 1 3


The -m switch defines access rights to the device file; c - indicates that the device is symbolic, and the subsequent digits are the major and minor number of the device. The first characterizes the type of character device, the second - the number of the device in the system.

/ dev / console provides the system with a terminal, / dev / null is most often used to get data from an empty file, or to redirect output to nowhere.

Now we mount the virtual file system in a special way. Such a mount is often called “binding” - when it is executed, the contents of the mounted directory are mapped to the directory specified as the mount point

# mount -v --bind /dev $LFS/dev


Now our temporary system will have access to device files. Mount the remaining VFS hosts

# mount -vt devpts devpts $LFS/dev/pts -o gid=5,mode=620
# mount -vt proc proc $LFS/proc
# mount -vt sysfs sysfs $LFS/sys
# mount -vt tmpfs tmpfs $LFS/run


/ dev / pts is a file system that provides access to pseudo-terminals. It is mounted with the identifier gid = 5 of the tty group with access rights 620 (-rw- -r- ---).

/ proc is a virtual file system containing information about the processes running on the system.

/ sys - a virtual file system that adds information about devices and drivers present in the system to user space

/ run - a virtual file system designed to create temporary files in the early stages of system boot. Previously, similar files were created in / dev, however, on the initiative of Lenard Pottering, in connection with the promotion of systemd, an additional directory was added for this purpose.

In addition, on some host systems in / dev, there may be symbolic links to / run / shm - temporary files for accessing shared memory. We check their availability on our host, and if necessary, create in the temporary system

$ if [ -h $LFS/dev/shm ]; then
>   mkdir -pv $LFS/$(readlink $LFS/dev/shm)
> fi


Now after the “devil” we get the structure schematically shown in the following figure.

LFS directory tree in relation to the host system


Actually, we are now logging into the temporary system

# chroot "$LFS" /tools/bin/env -i \
>    HOME=/root                  \
>    TERM="$TERM"                \
>    PS1='\u:\w\$ '              \
>    PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin \
>    /tools/bin/bash --login +h


The command changes the root directory of the system to $ LFS and executes two commands - / tools / bin / env - creates a new "clean" user environment, assigning some variables: HOME - position of the home directory; TERM - terminal type; PS1 - command line prompt format; PATH - a list of paths to executable files; / tools / bin / bash --login + h - launches a new instance of the shell, and the launch is performed as an entrance to a separate user session with disabling hashing of paths to executable files. Note that we use the programs (env and bash) that we built in the previous step.

After executing the commands, we get the following result
I have no name!:/# ls
dev  proc  run	sources  sys  tools

The root of the system has successfully changed, and the command prompt prompts you with a warning “I don’t have a name!”, Which occurs due to the absence of the / etc / passwd file in the system. Let this not bother you, we will create a file soon.

Note : System assembly can take a long time, so the VFS and chroot mount commands can be combined into a script in order to simplify entry to the system being built after turning off the computer.

2. Creating the necessary directories and files


Now we need to create the directory tree of our system

# mkdir -pv /{bin,boot,etc/{opt,sysconfig},home,lib/firmware,mnt,opt}
# mkdir -pv /{media/{floppy,cdrom},sbin,srv,var}
# install -dv -m 0750 /root
# install -dv -m 1777 /tmp /var/tmp
# mkdir -pv /usr/{,local/}{bin,include,lib,sbin,src}
# mkdir -pv /usr/{,local/}share/{color,dict,doc,info,locale,man}
# mkdir -v  /usr/{,local/}share/{misc,terminfo,zoneinfo}
# mkdir -v  /usr/libexec
# mkdir -pv /usr/{,local/}share/man/man{1..8}
# case $(uname -m) in
> x86_64) ln -sv lib /lib64
>         ln -sv lib /usr/lib64
>         ln -sv lib /usr/local/lib64 ;;
> esac
# mkdir -v /var/{log,mail,spool}
# ln -sv /run /var/run
# ln -sv /run/lock /var/lock
# mkdir -pv /var/{opt,cache,lib/{color,misc,locate},local}


Directories created by the mkdir command default to 755 (drwx rx rx) permissions. The install command is used in this case to create directories with specific access attributes: 0750 (drwx rx ---) for / root - the root directory of the superuser; 1777 (drwx rwx rwx) for / tmp and / var / tmp - directories for temporary files, which should be accessible to all users of the system, without exception. However, a ban on moving another user's files contained in the temporary directory is required. To do this, the so-called sticky bit is cocked, indicating that deleting files in the temporary directory is allowed only to their owner.

If you are building a 64-bit system, you must create symbolic links leading to the / lib directory containing the system libraries. In addition, some programs use hard-coded paths to the necessary components of the system. While we are working in a temporary system, these paths will be different (for example, all executable files are located in the / tools / bin directory so far) so you need to create symbolic links to fix this discrepancy

# ln -sv /tools/bin/{bash,cat,echo,pwd,stty} /bin
# ln -sv /tools/bin/perl /usr/bin
# ln -sv /tools/lib/libgcc_s.so{,.1} /usr/lib
# ln -sv /tools/lib/libstdc++.so{,.6} /usr/lib
# sed 's/tools/usr/' /tools/lib/libstdc++.la > /usr/lib/libstdc++.la
# ln -sv bash /bin/sh


Traditionally, in Linux, the list of mounted file systems was located in the / etc / mtab file. However, modern kernels use the / proc virtual file system for this. Create a symbolic link

# ln -sv /proc/self/mounts /etc/mtab


for programs that continue to use / etc / mtab.

Now we need to create the / etc / passwd file containing the list of users registered in the system.

# cat > /etc/passwd << "EOF"
> root:x:0:0:root:/root:/bin/bash
> bin:x:1:1:bin:/dev/null:/bin/false
> daemon:x:6:6:Daemon User:/dev/null:/bin/false
> messagebus:x:18:18:D-Bus Message Daemon User:/var/run/dbus:/bin/false
> nobody:x:99:99:Unprivileged User:/dev/null:/bin/false
> EOF


using the redirection of standard input to a file for this, since in the temporary system we do not have a text editor. This text file has the following format - each line contains information about the user in the form

::::::


  • login - username in the system
  • password - a password hash previously stored here, but now moved to a separate file / etc / shadow, inaccessible for reading by a normal user. Instead, the placeholder "x" is now placed
  • UID - user identifier - a number from 0 to 2 32 - 1. Actually, the main purpose of this file is to compare the login and user identifier.
  • GID - identifier of the default group in which the user is included
  • GECOS - an information field that stores additional information about the user. It does not have a clear syntax and, in fact, may contain any comments on this account.
  • home - absolute path to the user's home directory
  • shell - a command shell used in a given user's session


A colon is used as a field separator. Our file contains a description of the root superuser, as well as special users bin, daemon, messagebus on behalf of which some programs and services are launched.

Of particular importance is the unprivileged user nobody - on his behalf we will run tests of some of the assembled system components. Since we do not yet have access to the useradd utility of this user, as well as others, we create it manually.

Manually create the necessary set of user groups in the file / etc / group

# cat > /etc/group << "EOF"
> root:x:0:
> bin:x:1:daemon
> sys:x:2:
> kmem:x:3:
> tape:x:4:
> tty:x:5:
> daemon:x:6:
> floppy:x:7:
> disk:x:8:
> lp:x:9:
> dialout:x:10:
> audio:x:11:
> video:x:12:
> utmp:x:13:
> usb:x:14:
> cdrom:x:15:
> adm:x:16:
> messagebus:x:18:
> systemd-journal:x:23:
> input:x:24:
> mail:x:34:
> nogroup:x:99:
> users:x:999:
> EOF


Each line describes a separate group in the format

:::


  • group name - the symbolic name of the group
  • group passwd - group password - contains "x" because this field is outdated and is not currently used
  • GID - group identifier
  • users list - comma separated user list


Briefly describe the purpose of each of the groups

  • root is a group of superusers with root privileges. Under normal conditions, only the root user is included.
  • bin, sys - these groups were saved as historically existing, some programs do not work if they are not in the system
  • kmem - kernel memory access group
  • tape - group of access to device nodes
  • tty - access group to terminal devices (including the console)
  • daemon - unprivileged daemon startup group
  • floppy, cdrom, disk - access groups for various types of disk drives
  • lp - access group for the printer, historically "sitting" on the parallel port of LPT.
  • dialout - a group of access to devices that perform “dial-up” during a network connection (modems)
  • audio, video - access to multimedia devices
  • usb - USB bus access group
  • adm - access group for viewing system logs in the / var / log directory
  • messagebus - access group to the message bus between D-BUS devices
  • systemd-journal - access to systemd logs
  • input - access to user input devices
  • mail - access to email services
  • nogroup - empty group
  • users - regular, unprivileged, system users


You can now re-enter the user session

exec /tools/bin/bash --login +h


and, voila, now the system knows the name of the user on behalf of whom the login is made

root:/# 


We initialize some log files necessary for the system, assigning the appropriate rights

root:/#  touch /var/log/{btmp,lastlog,wtmp}
root:/#  chgrp -v utmp /var/log/lastlog
root:/#  chmod -v 664  /var/log/lastlog
root:/#  chmod -v 600  /var/log/btmp


3. The long and tedious process of building the GNU environment ...


This stage is the most laborious and lengthy. You have to manually collect all the necessary packages of the GNU environment - that minimal set of programs that allows you to get a working linux-system.

I must say, as you move along the list of packages you are collecting, nervous tension builds up - the list is quite extensive, and the operations are fairly uniform. At some point, the question arises in my head - what the hell do I need it at all ??? The main thing here is to collect thoughts and effort of will to overcome despondency - everything that has been started must be brought to an end.

The package assembly sequence is built so that at the current stage all the dependencies necessary for its execution are satisfied. So the assembly order must not be violated.

The assembly of each package is accompanied by detailed instructions, which provide in the general case

  • applying patches to the package source code. All necessary patches are downloaded by us and located in the / sources directory
  • source editing performed by sed utility. Here it is advisable to delve into the meaning of regular expressions that define the editing pattern, since thoughtless input of commands will probably lead to errors. And sed does not display error messages - if you make a mistake when entering the template, then the operation may not be executed or will be executed incorrectly, which will be fatal for the build process.
  • configuration Special attention to directory prefixes - during installation, all executable binaries, libraries, and documentation should be in place.
  • copying binary files, assigning rights to files and directories, creating symbolic links. Some links can lead to FS points that do not exist yet, so the syntax of the symlink creation commands should be understood literally and not allow amateur actions when constructing paths. Follow the instructions strictly!


And finally, the most important thing is tests! When assembling the temporary system, we ignored the tests, firstly because testing tools were not available to us, and secondly, there was no need for tests. Now, when we collect the final versions of the system software, we must comprehensively check the result of the assembly for proper functioning. This is especially true for the compiler, standard libraries and auxiliary tools for building software (assembler and linker).

Do not skip testing if there is a note in the instructions that it should be performed. Testing is time-consuming, for example, GCC assembly with testing takes place within 63 SBUs, which on my machine is about 150 minutes. But we have no other choice.

The manual contains notes on which tests may fail to pass with reasons. After completing the tests, we mandatory verify the result with the expected result according to the instructions. The reference logs of all tests for version LFS 7.7 are located at this link.

As an example, I will give an extract from my testing log GCC

GCC Brief Test Log
=== g ++ Summary ===

# of expected passes 88501
# of unexpected successes 2
# of expected failures 443
# of unsupported tests 3058
/sources/gcc-build/gcc/testsuite/g++/../../xg++ version 4.9 .2 (GCC)

- === gcc Summary ===

# of expected passes 106352
# of expected failures 252
# of unsupported tests 1422
/ sources / gcc-build / gcc / xgcc version 4.9.2 (GCC)

=== libatomic tests ===
- === libatomic Summary ===

# of expected passes 54
=== libgomp tests ===

Running target unix

=== libgomp Summary ===

# of expected passes 693
=== libitm tests ===

Running target unix

=== libitm Summary ===

# of expected passes 26
# of expected failures 3
# of unsupported tests 1
=== libstdc ++ tests ===

- === libstdc ++ Summary ===

# of expected passes 9925
# of expected failures 41
# of unsupported tests 233

Compiler version: 4.9.2 (GCC)
Platform: x86_64-unknown-linux-gnu


which can be compared with a sample

The official GCC build log from LFS authors
=== gcc Summary ===

# of expected passes 106401
# of expected failures 252
# of unsupported tests 1404
/ sources / gcc-build / gcc / xgcc version 4.9.2 (GCC)

make [4]: ​​Leaving directory '/ sources / gcc-build / gcc '
- === g ++ Summary ===

# of expected passes 88501
# of unexpected successes 2
# of expected failures 443
# of unsupported tests 3058
/ sources / gcc-build / gcc / testsuite / g ++ /. ./../xg++ version 4.9.2 (GCC)

- === libstdc ++ Summary ===

# of expected passes 9835
# of expected failures 41
# of unsupported tests 278
make [5]: Leaving directory '/ sources / gcc- build / x86_64-unknown-linux-gnu / libstdc ++ - v3 / testsuite '
make [4]: ​​Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libstdc ++ - v3 / testsuite'
Making check in python
- === libgomp Summary ===

# of expected passes 693
make [5 ]: Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libgomp / testsuite'
make [4]: ​​Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libgomp / testsuite '
make [4]: ​​Entering directory' / sources / gcc-build / x86_64-unknown-linux-gnu / libgomp '
true DO = all multi-do # make
make [4]: ​​Leaving directory' / sources / gcc-build / x86_64-unknown-linux-gnu / libgomp '
- === libitm Summary ===

# of expected passes 26
# of expected failures 3
# of unsupported tests 1
make [5]: Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libitm / testsuite'
make [4]: ​​Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libitm / testsuite '
make [4]: ​​Entering directory' / sources / gcc-build / x86_64-unknown-linux-gnu / libitm '
- === libatomic Summary ===

# of expected passes 54
make [5]: Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libatomic / testsuite'
make [4]: ​​Leaving directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libatomic / testsuite'
make [4 ]: Entering directory '/ sources / gcc-build / x86_64-unknown-linux-gnu / libatomic'
true DO = all multi-do # make
make [4]: ​​Leaving directory '/ sources / gcc-build / x86_64-unknown- linux-gnu / libatomic '
- === g ++ Summary ===

# of expected passes 88501
# of unexpected successes 2
# of expected failures 443
# of unsupported tests 3058
/sources/gcc-build/gcc/testsuite/g++/../../xg++ version 4.9.2 (GCC)

- === gcc Summary ===

# of expected passes 106401
# of expected failures 252
# of unsupported tests 1404
/ sources / gcc-build / gcc / xgcc version 4.9.2 (GCC)

=== libatomic tests ===
- === libatomic Summary ===

# of expected passes 54
=== libgomp tests ===

Running target unix

=== libgomp Summary ===

# of expected passes 693
=== libitm tests == =

Running target unix

=== libitm Summary ===

# of expected passes 26
# of expected failures 3
# of unsupported tests 1
=== libstdc ++ tests ===

- === libstdc ++ Summary ===

# of expected passes 9835
# of expected failures 41
# of unsupported tests 278

Compiler version: 4.9.2 (GCC)
Platform: x86_64-unknown-linux-gnu


I was lucky - I got results that coincide well with the proposed authors. Which I wish you too.

4. Re-entering the system, clearing debug information, deleting the temporary system


So, you have collected all the packages. Congratulations - the longest and most tedious part of the journey was covered by you. Now log in to our new system

# logout
# chroot $LFS /tools/bin/env -i            \
>    HOME=/root TERM=$TERM PS1='\u:\w\$ ' \
>    PATH=/bin:/usr/bin:/sbin:/usr/sbin   \
>   /tools/bin/bash --login


restarting / tools / bin / bash with hash enabled, and removing the / tools / bin directory from the paths to search for executable files. We cut off debugging information

# /tools/bin/find /{,usr/}{bin,lib,sbin} -type f \
>    -exec /tools/bin/strip --strip-debug '{}' ';'


We clean the temporary directory from the files remaining after testing

# rm -rf /tmp/*


log in again, launching the already "fighting" bash

# chroot "$LFS" /usr/bin/env -i              \
>    HOME=/root TERM="$TERM" PS1='\u:\w\$ ' \
>    PATH=/bin:/usr/bin:/sbin:/usr/sbin     \
>    /bin/bash --login


Now we no longer need a temporary system - delete it

# rm -rf /tools


Fuh ... the main difficulties are behind. It remains for us to configure our system to boot, build the kernel, configure the network. But I will write about this in the final article.

The ending follows ...

Also popular now: