aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--doc/news.rst10
-rw-r--r--doc/tutorial/disk_image.rst1
-rw-r--r--doc/tutorial/os_installation.rst58
-rw-r--r--src/package.lisp5
-rw-r--r--src/property/disk.lisp113
-rw-r--r--src/property/installer.lisp85
6 files changed, 203 insertions, 69 deletions
diff --git a/doc/news.rst b/doc/news.rst
index c85fe68..da2ecde 100644
--- a/doc/news.rst
+++ b/doc/news.rst
@@ -26,6 +26,16 @@ you should review this document and see if your consfig needs updating.
1.1.0 (unreleased)
------------------
+- API change: DISK:HOST-VOLUMES-CREATED has been removed in favour of new
+ properties DISK:FIRST-DISK-INSTALLED-FOR and DISK:VOLUMES-INSTALLED-FOR.
+
+- API change: INSTALLED:CHROOT-INSTALLED-TO-VOLUMES-FOR has been renamed to
+ INSTALLER:FILES-INSTALLED-TO-VOLUMES-FOR, and will now bootstrap a root
+ filesystem directly to the volumes if not supplied a chroot. The CHROOT
+ parameter has become a keyword parameter, and the required parameters have
+ changed from ``(HOST CHROOT VOLUMES)`` to ``(OPTIONS HOST VOLUMES)`` for
+ consistency with other property lambda lists.
+
- API change: DISK:WITH-OPENED-VOLUMES now includes volumes that were already
open, and their parents, in the connattrs.
diff --git a/doc/tutorial/disk_image.rst b/doc/tutorial/disk_image.rst
index cf22cd3..d4b7a20 100644
--- a/doc/tutorial/disk_image.rst
+++ b/doc/tutorial/disk_image.rst
@@ -15,6 +15,7 @@ Here is a minimal definition of the host for which we can build a disk image:::
(os:debian-stable "bullseye" :amd64)
(disk:has-volumes
(physical-disk
+ :device-file #P"/dev/sda"
:boots-with '(grub:grub :target "x86_64-efi" :force-extra-removable t)
(partitioned-volume
((partition
diff --git a/doc/tutorial/os_installation.rst b/doc/tutorial/os_installation.rst
index f15da41..d89ccbf 100644
--- a/doc/tutorial/os_installation.rst
+++ b/doc/tutorial/os_installation.rst
@@ -1,8 +1,7 @@
Tutorial: OS installation
=========================
-Consfigurator implements at least the basic elements of a number of methods
-for installing operating systems.
+Consfigurator implements a number of methods for installing operating systems.
.. include:: conventions.rst
@@ -17,6 +16,17 @@ expand the partitions to fill the whole physical disk after the first
successful boot, so the disk image has to be the same size as the target disk,
which can be unwieldy.
+Installing directly to target host's primary storage
+----------------------------------------------------
+
+This is similar to the previous method, but it avoids the problem of having to
+expand filesystems upon first boot. Suppose we have connected the target
+host's hard disk to our laptop, or transferred its boot SD card to our
+laptop's SD card reader, etc.. Then we can install to it directly:::
+
+ CONSFIG> (hostdeploy-these laptop.example.com
+ (disk:first-disk-installed-for nil test.example.com "/dev/mmcblk0"))
+
Live replacement of provider cloud images
-----------------------------------------
@@ -29,36 +39,43 @@ has been taught how to bootstrap them.
Build a specialised live image
------------------------------
-This third approach is more experimental; Consfigurator has all the necessary
-capabilities, at least for Debian, but at present you'll need to string them
-together yourself in your consfig. With this approach you build a live image
-containing everything you need to run Consfigurator on the hardware to which
-you want to install. After booting up the live system, you can either run
-Consfigurator manually, or you can set things up to have it run automatically
-upon boot.
+This fourth approach requires more work in your consfig. With this approach
+you build a live image containing everything you need to run Consfigurator on
+the hardware to which you want to install. After booting up the live system,
+you can either run Consfigurator manually, or you can set things up to have it
+run automatically upon boot.
Consfigurator's ability to bootstrap fresh root filesystems typically requires
Internet access, but an alternative is to build and customise a chroot
corresponding to the root filesystem of the target system, and include that in
the live image, such that after boot Consfigurator just needs to partition the
disk, copy in the contents of the prebuilt chroot, and update /etc/fstab and
-/etc/crypttab with UUIDs. Here is a sketch of how to do something like that::
+/etc/crypttab with UUIDs. Here is a minimal version of something like that::
(try-register-data-source
:git-snapshot :name "consfig"
- :repo #P"src/cl/consfig/" :depth 1 :branch "master")
+ :repo #P"common-lisp/consfig/" :depth 1 :branch "master")
(defproplist live-installer-built-for :lisp (with-chroot-for)
"Build a custom Debian Live system at /srv/live/installer.iso.
Typically this property is not applied in a DEFHOST form, but rather run as
needed at the REPL. The reason for this is that otherwise the whole image will
- get rebuilt each time a commit is made to ~/src/cl/consfig/."
+ get rebuilt each time a commit is made to ~/common-lisp/consfig/."
(:desc "Debian Live system image built")
(disk:debian-live-iso-built. nil "/srv/live/installer.iso"
(os:debian-stable "bullseye" :amd64)
- (apt:installed "task-english" "live-config" "lvm2" "cryptsetup")
- (git:snapshot-extracted "consfig" "/etc/skel/src/cl")
+ (hostname:configured "debian")
+ (apt:installed "sudo" "task-english" "emacs"
+ "sbcl" "slime" "cl-consfigurator" "build-essential"
+ "lvm2" "cryptsetup" "gdisk" "kpartx" "dosfstools")
+
+ (user:has-account "user")
+ (user:has-enabled-password "user" :initial-password "live")
+ (file:exists-with-content "/etc/sudoers.d/user"
+ '("user ALL=(ALL:ALL) NOPASSWD: ALL") :mode #o600)
+ (as "user" (git:snapshot-extracted "consfig" "common-lisp/" :replace t))
+
(chroot:os-bootstrapped-for
nil
(merge-pathnames (get-hostname with-chroot-for) "/srv/chroot/")
@@ -70,9 +87,16 @@ could then use::
CONSFIG> (hostdeploy-these laptop.example.com
(live-installer-built-for test.example.com))
-Then once the live system has booted on the target host, you'd use the
-DISK:HOST-VOLUMES-CREATED and INSTALLER:CHROOT-INSTALLED-TO-VOLUMES-FOR
-properties to complete the installation.
+Then once the live system has booted on the target host, you'd complete the
+installation using something like this:::
+
+ CONSFIG> (deploy-these (:sudo :sbcl) "debian"
+ (os:debian-stable "bullseye" :amd64)
+ (disk:volumes-installed-for nil test.example.com :leave-open t
+ :chroot "/srv/chroot/test.example.com")
+
+This approach requires that a ``:DEVICE-FILE`` is specified for each of the
+host's physical disks.
To prepare a live image that is capable of installing more than one system
without an Internet connection, you'd probably need to investigate including
diff --git a/src/package.lisp b/src/package.lisp
index 69b50fb..e6f0362 100644
--- a/src/package.lisp
+++ b/src/package.lisp
@@ -651,9 +651,10 @@
#:has-volumes
#:raw-image-built-for
+ #:first-disk-installed-for
+ #:volumes-installed-for
#:debian-live-iso-built
#:debian-live-iso-built.
- #:host-volumes-created
#:host-logical-volumes-exist
#:volumes))
@@ -741,7 +742,7 @@
(#:disk #:consfigurator.property.disk))
(:export #:install-bootloader-propspec
#:install-bootloader-binaries-propspec
- #:chroot-installed-to-volumes-for
+ #:files-installed-to-volumes-for
#:bootloader-binaries-installed
#:bootloaders-installed
#:cleanly-installed-once
diff --git a/src/property/disk.lisp b/src/property/disk.lisp
index af1e7e6..c1759a0 100644
--- a/src/property/disk.lisp
+++ b/src/property/disk.lisp
@@ -978,8 +978,108 @@ filesystems will be incrementally updated when other properties change."
(caches-cleaned)))
(%raw-image-created
,volumes :chroot ,chroot :rebuild ,rebuild))
- (consfigurator.property.installer:chroot-installed-to-volumes-for
- ,host ,chroot ,volumes))))
+ (consfigurator.property.installer:files-installed-to-volumes-for
+ nil ,host ,volumes :chroot ,chroot))))
+
+(defprop %volumes-created :lisp (volumes)
+ (:desc "Created host volumes")
+ (:hostattrs (os:required 'os:linux) (require-volumes-data volumes))
+ (:apply
+ (dolist (volume volumes)
+ (when (subtypep (type-of volume) 'physical-disk)
+ (setf (volume-size volume)
+ (floor (/ (parse-integer
+ (stripln
+ (run "blockdev" "--getsize64" (device-file volume))))
+ 1048576)))))
+ (create-volumes-and-contents volumes)))
+
+(defpropspec first-disk-installed-for :lisp
+ (options host device-file &key chroot)
+ "Install HOST to the DISK:PHYSICAL-DISK accessible at DEVICE-FILE.
+**THIS PROPERTY UNCONDITIONALLY FORMATS DISKS, POTENTIALLY DESTROYING DATA,
+ EACH TIME IT IS APPLIED.**
+
+Do not apply in DEFHOST. Apply with DEPLOY-THESE/HOSTDEPLOY-THESE.
+
+The DISK:VOLUME-CONTENTS of the first DISK:PHYSICAL-DISK entry in the host's
+volumes, as specified using DISK:HAS-VOLUMES, will be created at DEVICE-FILE;
+there must be at least one such DISK:PHYSICAL-DISK entry. Other
+DISK:PHYSICAL-DISK entries will be ignored, so ensure that none of the
+properties of the host will write to areas of the filesystem where filesystems
+stored on other physical disks would normally be mounted.
+
+OPTIONS will be passed on to CHROOT:OS-BOOTSTRAPPED-FOR, which see.
+
+In most cases you will need to ensure that HOST has properties which do at
+least the following:
+
+ - declare the host's OS
+
+ - install a kernel
+
+ - install the binaries/packages needed to install the host's bootloader to
+ its volumes (usually with INSTALLER:BOOTLOADER-BINARIES-INSTALLED).
+
+If CHROOT, an intermediate chroot is bootstrapped at CHROOT, and HOST's
+properties are applied to that. Otherwise, HOST's OS is bootstrapped directly
+to DEVICE-FILE. It's useful to supply CHROOT when you expect to install the
+same HOST to a number of physical disks.
+
+Applying this property is similar to applying DISK:RAW-IMAGE-BUILT-FOR and
+then immediately dd'ing out the image to DEVICE-FILE. The advantage of this
+property is that there is no need to resize filesystems to fill the size of
+the host's actual physical disk upon first boot."
+ (:desc #?"Installed ${(get-hostname host)} to ${device-file}")
+ (let ((volumes (host-volumes-just-one-physical-disk
+ host (lambda (v)
+ (setf (device-file v)
+ (parse-unix-namestring device-file))))))
+ `(eseqprops
+ (%volumes-created ,volumes)
+ ,@(and chroot
+ (list (propapp (chroot:os-bootstrapped-for. options chroot host
+ (caches-cleaned)))))
+ (consfigurator.property.installer:files-installed-to-volumes-for
+ ,options ,host ,volumes :chroot ,chroot))))
+
+(defpropspec volumes-installed-for :lisp (options host &key chroot leave-open)
+ "Install HOST to its volumes, as specified using DISK:HAS-VOLUMES.
+**THIS PROPERTY UNCONDITIONALLY FORMATS DISKS, POTENTIALLY DESTROYING DATA,
+ EACH TIME IT IS APPLIED.**
+
+Do not apply in DEFHOST. Apply with DEPLOY-THESE/HOSTDEPLOY-THESE.
+
+OPTIONS will be passed on to CHROOT:OS-BOOTSTRAPPED-FOR, which see.
+
+In most cases you will need to ensure that HOST has properties which do at
+least the following:
+
+ - declare the host's OS
+
+ - install a kernel
+
+ - install the binaries/packages needed to install the host's bootloader to
+ its volumes (usually with INSTALLER:BOOTLOADER-BINARIES-INSTALLED).
+
+If CHROOT, an intermediate chroot is bootstrapped at CHROOT, and HOST's
+properties are applied to that. Otherwise, HOST's OS is bootstrapped directly
+to its volumes. This parameter is useful for the case of installing HOST from
+a live system which might not have network access. See \"Tutorial: OS
+installation\" in the Consfigurator user's manual.
+
+If LEAVE-OPEN, HOST's volumes will not be closed. This allows you to inspect
+the result of the installation. If you want to run this property again, you
+should first manually close all the volumes."
+ (:desc #?"Installed ${(get-hostname host)}")
+ (let ((volumes (get-hostattrs :volumes host)))
+ `(eseqprops
+ (%volumes-created ,volumes)
+ ,@(and chroot
+ (list (propapp (chroot:os-bootstrapped-for. options chroot host
+ (caches-cleaned)))))
+ (consfigurator.property.installer:files-installed-to-volumes-for
+ ,options ,host ,volumes :chroot ,chroot :leave-open ,leave-open))))
(defprop %squashfsed :posix (chroot image &optional (compression "xz"))
(:apply
@@ -1086,15 +1186,6 @@ Currently only BIOS boot is implemented."
,iso-root)))))
-(defprop host-volumes-created :lisp ()
- "Recursively create the volumes as specified by DISK:HAS-VOLUMES.
-**THIS PROPERTY UNCONDITIONALLY FORMATS DISKS, POTENTIALLY DESTROYING DATA,
- EACH TIME IT IS APPLIED.**
-
-Do not apply in DEFHOST. Apply with DEPLOY-THESE/HOSTDEPLOY-THESE."
- (:desc "Host volumes created")
- (:apply (create-volumes-and-contents (get-hostattrs :volumes))))
-
;; TODO Possibly we want (a version of) this to not fail, but just do nothing,
;; if the relevant volume groups etc. are inactive?
(defproplist host-logical-volumes-exist :lisp ()
diff --git a/src/property/installer.lisp b/src/property/installer.lisp
index b70e2a1..e2d8585 100644
--- a/src/property/installer.lisp
+++ b/src/property/installer.lisp
@@ -59,11 +59,11 @@ BOOTLOADER-TYPE to VOLUME."))
;; At :HOSTATTRS time we don't have the OPENED-VOLUME values required by the
;; :APPLY subroutines which actually install the bootloaders. So we call
-;; GET-PROPSPECS twice: (in CHROOT-INSTALLED-TO-VOLUMES-FOR) at :HOSTATTRS
-;; time to generate propspecs for the sake of running :HOSTATTRS subroutines,
-;; and then at :APPLY time where we can get at the OPENED-VOLUME values, we
-;; ignore the previously generated propspecs and call GET-PROPSPECS again.
-;; This approach should work for any sensible VOLUME<->OPENED-VOLUME pairs.
+;; GET-PROPSPECS twice: (in FILES-INSTALLED-TO-VOLUMES-FOR) at :HOSTATTRS time
+;; to generate propspecs for the sake of running :HOSTATTRS subroutines, and
+;; then at :APPLY time where we can get at the OPENED-VOLUME values, we ignore
+;; the previously generated propspecs and call GET-PROPSPECS again. This
+;; approach should work for any sensible VOLUME<->OPENED-VOLUME pairs.
(define-function-property-combinator %install-bootloaders (&rest propapps)
(:retprop
:type :lisp
@@ -89,44 +89,51 @@ BOOTLOADER-TYPE to VOLUME."))
(strcat (unix-namestring chroot) "/")
(strcat (unix-namestring target) "/"))))
-(defpropspec chroot-installed-to-volumes-for :lisp
- (host chroot volumes &key leave-open)
+(defun chroot-target (chroot)
+ (ensure-directory-pathname
+ (strcat
+ (drop-trailing-slash (unix-namestring (ensure-directory-pathname chroot)))
+ ".target")))
+
+(defpropspec files-installed-to-volumes-for :lisp
+ (options host volumes &key chroot leave-open
+ (mount-below (if chroot (chroot-target chroot) #P"/target/")))
"Where CHROOT contains the root filesystem of HOST and VOLUMES is a list of
volumes, recursively open the volumes and rsync in the contents of CHROOT.
Also update the fstab and crypttab, and try to install bootloader(s).
-LEAVE-OPEN is passed on to DISK:WITH-OPENED-VOLUMES, which see."
- (:desc #?"${chroot} installed to volumes")
- (let ((target
- (ensure-directory-pathname
- (strcat
- (drop-trailing-slash
- (unix-namestring (ensure-directory-pathname chroot)))
- ".target"))))
- `(with-opened-volumes
- (,volumes :mount-below ,target :leave-open ,leave-open)
- (%update-target-from-chroot ,chroot ,target)
- (chroot:deploys-these
- ,target ,host
- ,(make-propspec
- :propspec
- `(eseqprops
- ,(propapp
- (os:etypecase
- (debianlike
- (file:lacks-lines
- "/etc/fstab" "# UNCONFIGURED FSTAB FOR BASE SYSTEM")
- ;; These will overwrite any custom mount options, etc.,
- ;; with values from VOLUMES. Possibly it would be better
- ;; to use properties which only update the fs-spec/source
- ;; fields. However, given that VOLUMES ultimately comes
- ;; from the volumes the user has declared for the host, it
- ;; is unlikely there are other properties setting mount
- ;; options etc. which are in conflict with VOLUMES.
- (fstab:has-entries-for-opened-volumes)
- (crypttab:has-entries-for-opened-volumes))))
- (%install-bootloaders
- ,@(get-propspecs (get-hostattrs :volumes)))))))))
+If CHROOT is NIL, bootstrap a root filesystem for HOST directly to VOLUMES.
+In that case, OPTIONS is passed on to CHROOT:OS-BOOTSTRAPPED-FOR, which see.
+
+MOUNT-BELOW and LEAVE-OPEN are passed on to WITH-OPENED-VOLUMES, which see."
+ (:desc (format nil "~A installed to volumes"
+ (or chroot (get-hostname host))))
+ `(with-opened-volumes
+ (,volumes :mount-below ,mount-below :leave-open ,leave-open)
+ ,(if chroot
+ `(%update-target-from-chroot ,chroot ,mount-below)
+ `(chroot:os-bootstrapped-for ,options ,mount-below ,host))
+ (chroot:deploys-these
+ ,mount-below ,host
+ ,(make-propspec
+ :propspec
+ `(eseqprops
+ ,(propapp
+ (os:etypecase
+ (debianlike
+ (file:lacks-lines
+ "/etc/fstab" "# UNCONFIGURED FSTAB FOR BASE SYSTEM")
+ ;; These will overwrite any custom mount options, etc., with
+ ;; values from VOLUMES. Possibly it would be better to use
+ ;; properties which only update the fs-spec/source fields.
+ ;; However, given that VOLUMES ultimately comes from the
+ ;; volumes the user has declared for the host, it is unlikely
+ ;; there are other properties setting mount options etc. which
+ ;; are in conflict with VOLUMES.
+ (fstab:has-entries-for-opened-volumes)
+ (crypttab:has-entries-for-opened-volumes))))
+ (%install-bootloaders
+ ,@(get-propspecs (get-hostattrs :volumes))))))))
(defpropspec bootloader-binaries-installed :posix ()
"Install whatever binaries/packages need to be available to install the host's
generated by cgit v1.2.3 (git 2.39.1) at 2024-04-19 05:26:48 +0000