| Design document for the unified scheme data |
| =========================================== |
| |
| How are things connected? |
| ------------------------- |
| |
| The unified scheme takes all its data from the build.info files seen |
| throughout the source tree. These files hold the minimum information |
| needed to build end product files from diverse sources. See the |
| section on build.info files below. |
| |
| From the information in build.info files, Configure builds up an |
| information database as a hash table called %unified_info, which is |
| stored in configdata.pm, found at the top of the build tree (which may |
| or may not be the same as the source tree). |
| |
| Configurations/common.tmpl uses the data from %unified_info to |
| generate the rules for building end product files as well as |
| intermediary files with the help of a few functions found in the |
| build-file templates. See the section on build-file templates further |
| down for more information. |
| |
| build.info files |
| ---------------- |
| |
| As mentioned earlier, build.info files are meant to hold the minimum |
| information needed to build output files, and therefore only (with a |
| few possible exceptions [1]) have information about end products (such |
| as scripts, library files and programs) and source files (such as C |
| files, C header files, assembler files, etc). Intermediate files such |
| as object files are rarely directly referred to in build.info files (and |
| when they are, it's always with the file name extension .o), they are |
| inferred by Configure. By the same rule of minimalism, end product |
| file name extensions (such as .so, .a, .exe, etc) are never mentioned |
| in build.info. Their file name extensions will be inferred by the |
| build-file templates, adapted for the platform they are meant for (see |
| sections on %unified_info and build-file templates further down). |
| |
| The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare |
| end products. There are variants for them with '_NO_INST' as suffix |
| (PROGRAM_NO_INST etc) to specify end products that shouldn't get |
| installed. |
| |
| The variables SOURCE, DEPEND and INCLUDE are indexed by a produced |
| file, and their values are the source used to produce that particular |
| produced file, extra dependencies, and include directories needed. |
| |
| All their values in all the build.info throughout the source tree are |
| collected together and form a set of programs, libraries, engines and |
| scripts to be produced, source files, dependencies, etc etc etc. |
| |
| Let's have a pretend example, a very limited contraption of OpenSSL, |
| composed of the program 'apps/openssl', the libraries 'libssl' and |
| 'libcrypto', an engine 'engines/ossltest' and their sources and |
| dependencies. |
| |
| # build.info |
| LIBS=libcrypto libssl |
| INCLUDE[libcrypto]=include |
| INCLUDE[libssl]=include |
| DEPEND[libssl]=libcrypto |
| |
| This is the top directory build.info file, and it tells us that two |
| libraries are to be built, the include directory 'include/' shall be |
| used throughout when building anything that will end up in each |
| library, and that the library 'libssl' depend on the library |
| 'libcrypto' to function properly. |
| |
| # apps/build.info |
| PROGRAMS=openssl |
| SOURCE[openssl]=openssl.c |
| INCLUDE[openssl]=.. ../include |
| DEPEND[openssl]=../libssl |
| |
| This is the build.info file in 'apps/', one may notice that all file |
| paths mentioned are relative to the directory the build.info file is |
| located in. This one tells us that there's a program to be built |
| called 'apps/openssl' (the file name extension will depend on the |
| platform and is therefore not mentioned in the build.info file). It's |
| built from one source file, 'apps/openssl.c', and building it requires |
| the use of '.' and 'include' include directories (both are declared |
| from the point of view of the 'apps/' directory), and that the program |
| depends on the library 'libssl' to function properly. |
| |
| # crypto/build.info |
| LIBS=../libcrypto |
| SOURCE[../libcrypto]=aes.c evp.c cversion.c |
| DEPEND[cversion.o]=buildinf.h |
| |
| GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" |
| DEPEND[buildinf.h]=../Makefile |
| DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm |
| |
| This is the build.info file in 'crypto', and it tells us a little more |
| about what's needed to produce 'libcrypto'. LIBS is used again to |
| declare that 'libcrypto' is to be produced. This declaration is |
| really unnecessary as it's already mentioned in the top build.info |
| file, but can make the info file easier to understand. This is to |
| show that duplicate information isn't an issue. |
| |
| This build.info file informs us that 'libcrypto' is built from a few |
| source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'. |
| It also shows us that building the object file inferred from |
| 'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it |
| also shows the possibility to declare how some files are generated |
| using some script, in this case a perl script, and how such scripts |
| can be declared to depend on other files, in this case a perl module. |
| |
| Two things are worth an extra note: |
| |
| 'DEPEND[cversion.o]' mentions an object file. DEPEND indexes is the |
| only location where it's valid to mention them |
| |
| Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as |
| seen from the top directory, no exception. |
| |
| # ssl/build.info |
| LIBS=../libssl |
| SOURCE[../libssl]=tls.c |
| |
| This is the build.info file in 'ssl/', and it tells us that the |
| library 'libssl' is built from the source file 'ssl/tls.c'. |
| |
| # engines/build.info |
| ENGINES=dasync |
| SOURCE[dasync]=e_dasync.c |
| DEPEND[dasync]=../libcrypto |
| INCLUDE[dasync]=../include |
| |
| ENGINES_NO_INST=ossltest |
| SOURCE[ossltest]=e_ossltest.c |
| DEPEND[ossltest]=../libcrypto.a |
| INCLUDE[ossltest]=../include |
| |
| This is the build.info file in 'engines/', telling us that two engines |
| called 'engines/dasync' and 'engines/ossltest' shall be built, that |
| dasync's source is 'engines/e_dasync.c' and ossltest's source is |
| 'engines/e_ossltest.c' and that the include directory 'include/' may |
| be used when building anything that will be part of these engines. |
| Also, both engines depend on the library 'libcrypto' to function |
| properly. ossltest is explicitly linked with the static variant of |
| the library 'libcrypto'. Finally, only dasync is being installed, as |
| ossltest is only for internal testing. |
| |
| When Configure digests these build.info files, the accumulated |
| information comes down to this: |
| |
| LIBS=libcrypto libssl |
| SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c |
| DEPEND[crypto/cversion.o]=crypto/buildinf.h |
| INCLUDE[libcrypto]=include |
| SOURCE[libssl]=ssl/tls.c |
| INCLUDE[libssl]=include |
| DEPEND[libssl]=libcrypto |
| |
| PROGRAMS=apps/openssl |
| SOURCE[apps/openssl]=apps/openssl.c |
| INCLUDE[apps/openssl]=. include |
| DEPEND[apps/openssl]=libssl |
| |
| ENGINES=engines/dasync |
| SOURCE[engines/dasync]=engines/e_dasync.c |
| DEPEND[engines/dasync]=libcrypto |
| INCLUDE[engines/dasync]=include |
| |
| ENGINES_NO_INST=engines/ossltest |
| SOURCE[engines/ossltest]=engines/e_ossltest.c |
| DEPEND[engines/ossltest]=libcrypto.a |
| INCLUDE[engines/ossltest]=include |
| |
| GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" |
| DEPEND[crypto/buildinf.h]=Makefile |
| DEPEND[util/mkbuildinf.pl]=util/Foo.pm |
| |
| |
| A few notes worth mentioning: |
| |
| LIBS may be used to declare routine libraries only. |
| |
| PROGRAMS may be used to declare programs only. |
| |
| ENGINES may be used to declare engines only. |
| |
| The indexes for SOURCE must only be end product files, such as |
| libraries, programs or engines. The values of SOURCE variables must |
| only be source files (possibly generated). |
| |
| INCLUDE and DEPEND shows a relationship between different files |
| (usually produced files) or between files and directories, such as a |
| program depending on a library, or between an object file and some |
| extra source file. |
| |
| When Configure processes the build.info files, it will take it as |
| truth without question, and will therefore perform very few checks. |
| If the build tree is separate from the source tree, it will assume |
| that all built files and up in the build directory and that all source |
| files are to be found in the source tree, if they can be found there. |
| Configure will assume that source files that can't be found in the |
| source tree (such as 'crypto/bildinf.h' in the example above) are |
| generated and will be found in the build tree. |
| |
| |
| The %unified_info database |
| -------------------------- |
| |
| The information in all the build.info get digested by Configure and |
| collected into the %unified_info database, divided into the following |
| indexes: |
| |
| depends => a hash table containing 'file' => [ 'dependency' ... ] |
| pairs. These are directly inferred from the DEPEND |
| variables in build.info files. |
| |
| engines => a list of engines. These are directly inferred from |
| the ENGINES variable in build.info files. |
| |
| generate => a hash table containing 'file' => [ 'generator' ... ] |
| pairs. These are directly inferred from the GENERATE |
| variables in build.info files. |
| |
| includes => a hash table containing 'file' => [ 'include' ... ] |
| pairs. These are directly inferred from the INCLUDE |
| variables in build.info files. |
| |
| install => a hash table containing 'type' => [ 'file' ... ] pairs. |
| The types are 'programs', 'libraries', 'engines' and |
| 'scripts', and the array of files list the files of |
| that type that should be installed. |
| |
| libraries => a list of libraries. These are directly inferred from |
| the LIBS variable in build.info files. |
| |
| programs => a list of programs. These are directly inferred from |
| the PROGRAMS variable in build.info files. |
| |
| rawlines => a list of build-file lines. These are a direct copy of |
| the BEGINRAW..ENDRAW lines in build.info files. Note: |
| only the BEGINRAW..ENDRAW section for the current |
| platform are copied, the rest are ignored. |
| |
| scripts => a list of scripts. There are directly inferred from |
| the SCRIPTS variable in build.info files. |
| |
| sources => a hash table containing 'file' => [ 'sourcefile' ... ] |
| pairs. These are indirectly inferred from the SOURCE |
| variables in build.info files. Object files are |
| mentioned in this hash table, with source files from |
| SOURCE variables, and AS source files for programs and |
| libraries. |
| |
| shared_sources => |
| a hash table just like 'sources', but only as source |
| files (object files) for building shared libraries. |
| |
| As an example, here is how the build.info files example from the |
| section above would be digested into a %unified_info table: |
| |
| our %unified_info = ( |
| "depends" => |
| { |
| "apps/openssl" => |
| [ |
| "libssl", |
| ], |
| "crypto/buildinf.h" => |
| [ |
| "Makefile", |
| ], |
| "crypto/cversion.o" => |
| [ |
| "crypto/buildinf.h", |
| ], |
| "engines/dasync" => |
| [ |
| "libcrypto", |
| ], |
| "engines/ossltest" => |
| [ |
| "libcrypto.a", |
| ], |
| "libssl" => |
| [ |
| "libcrypto", |
| ], |
| "util/mkbuildinf.pl" => |
| [ |
| "util/Foo.pm", |
| ], |
| }, |
| "engines" => |
| [ |
| "engines/dasync", |
| "engines/ossltest", |
| ], |
| "generate" => |
| { |
| "crypto/buildinf.h" => |
| [ |
| "util/mkbuildinf.pl", |
| "\"\$(CC)", |
| "\$(CFLAGS)\"", |
| "\"$(PLATFORM)\"", |
| ], |
| }, |
| "includes" => |
| { |
| "apps/openssl" => |
| [ |
| ".", |
| "include", |
| ], |
| "engines/ossltest" => |
| [ |
| "include" |
| ], |
| "libcrypto" => |
| [ |
| "include", |
| ], |
| "libssl" => |
| [ |
| "include", |
| ], |
| "util/mkbuildinf.pl" => |
| [ |
| "util", |
| ], |
| } |
| "install" => |
| { |
| "engines" => |
| [ |
| "engines/dasync", |
| ], |
| "libraries" => |
| [ |
| "libcrypto", |
| "libssl", |
| ], |
| "programs" => |
| [ |
| "apps/openssl", |
| ], |
| }, |
| "libraries" => |
| [ |
| "libcrypto", |
| "libssl", |
| ], |
| "programs" => |
| [ |
| "apps/openssl", |
| ], |
| "rawlines" => |
| [ |
| ], |
| "sources" => |
| { |
| "apps/openssl" => |
| [ |
| "apps/openssl.o", |
| ], |
| "apps/openssl.o" => |
| [ |
| "apps/openssl.c", |
| ], |
| "crypto/aes.o" => |
| [ |
| "crypto/aes.c", |
| ], |
| "crypto/cversion.o" => |
| [ |
| "crypto/cversion.c", |
| ], |
| "crypto/evp.o" => |
| [ |
| "crypto/evp.c", |
| ], |
| "engines/e_dasync.o" => |
| [ |
| "engines/e_dasync.c", |
| ], |
| "engines/dasync" => |
| [ |
| "engines/e_dasync.o", |
| ], |
| "engines/e_ossltest.o" => |
| [ |
| "engines/e_ossltest.c", |
| ], |
| "engines/ossltest" => |
| [ |
| "engines/e_ossltest.o", |
| ], |
| "libcrypto" => |
| [ |
| "crypto/aes.c", |
| "crypto/cversion.c", |
| "crypto/evp.c", |
| ], |
| "libssl" => |
| [ |
| "ssl/tls.c", |
| ], |
| "ssl/tls.o" => |
| [ |
| "ssl/tls.c", |
| ], |
| }, |
| ); |
| |
| As can be seen, everything in %unified_info is fairly simple suggest |
| of information. Still, it tells us that to build all programs, we |
| must build 'apps/openssl', and to build the latter, we will need to |
| build all its sources ('apps/openssl.o' in this case) and all the |
| other things it depends on (such as 'libssl'). All those dependencies |
| need to be built as well, using the same logic, so to build 'libssl', |
| we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the |
| latter... |
| |
| |
| Build-file templates |
| -------------------- |
| |
| Build-file templates are essentially build-files (such as Makefile on |
| Unix) with perl code fragments mixed in. Those perl code fragment |
| will generate all the configuration dependent data, including all the |
| rules needed to build end product files and intermediary files alike. |
| At a minimum, there must be a perl code fragment that defines a set of |
| functions that are used to generates specific build-file rules, to |
| build static libraries from object files, to build shared libraries |
| from static libraries, to programs from object files and libraries, |
| etc. |
| |
| generatesrc - function that produces build file lines to generate |
| a source file from some input. |
| |
| It's called like this: |
| |
| generatesrc(src => "PATH/TO/tobegenerated", |
| generator => [ "generatingfile", ... ] |
| generator_incs => [ "INCL/PATH", ... ] |
| generator_deps => [ "dep1", ... ] |
| incs => [ "INCL/PATH", ... ], |
| deps => [ "dep1", ... ], |
| intent => one of "libs", "dso", "bin" ); |
| |
| 'src' has the name of the file to be generated. |
| 'generator' is the command or part of command to |
| generate the file, of which the first item is |
| expected to be the file to generate from. |
| generatesrc() is expected to analyse and figure out |
| exactly how to apply that file and how to capture |
| the result. 'generator_incs' and 'generator_deps' |
| are include directories and files that the generator |
| file itself depends on. 'incs' and 'deps' are |
| include directories and files that are used if $(CC) |
| is used as an intermediary step when generating the |
| end product (the file indicated by 'src'). 'intent' |
| indicates what the generated file is going to be |
| used for. |
| |
| src2obj - function that produces build file lines to build an |
| object file from source files and associated data. |
| |
| It's called like this: |
| |
| src2obj(obj => "PATH/TO/objectfile", |
| srcs => [ "PATH/TO/sourcefile", ... ], |
| deps => [ "dep1", ... ], |
| incs => [ "INCL/PATH", ... ] |
| intent => one of "lib", "dso", "bin" ); |
| |
| 'obj' has the intended object file *without* |
| extension, src2obj() is expected to add that. |
| 'srcs' has the list of source files to build the |
| object file, with the first item being the source |
| file that directly corresponds to the object file. |
| 'deps' is a list of explicit dependencies. 'incs' |
| is a list of include file directories. Finally, |
| 'intent' indicates what this object file is going |
| to be used for. |
| |
| obj2lib - function that produces build file lines to build a |
| static library file ("libfoo.a" in Unix terms) from |
| object files. |
| |
| called like this: |
| |
| obj2lib(lib => "PATH/TO/libfile", |
| objs => [ "PATH/TO/objectfile", ... ]); |
| |
| 'lib' has the intended library file name *without* |
| extension, obj2lib is expected to add that. 'objs' |
| has the list of object files (also *without* |
| extension) to build this library. |
| |
| libobj2shlib - function that produces build file lines to build a |
| shareable object library file ("libfoo.so" in Unix |
| terms) from the corresponding static library file |
| or object files. |
| |
| called like this: |
| |
| libobj2shlib(shlib => "PATH/TO/shlibfile", |
| lib => "PATH/TO/libfile", |
| objs => [ "PATH/TO/objectfile", ... ], |
| deps => [ "PATH/TO/otherlibfile", ... ]); |
| |
| 'lib' has the intended library file name *without* |
| extension, libobj2shlib is expected to add that. |
| 'shlib' has the corresponding shared library name |
| *without* extension. 'deps' has the list of other |
| libraries (also *without* extension) this library |
| needs to be linked with. 'objs' has the list of |
| object files (also *without* extension) to build |
| this library. |
| |
| This function has a choice; it can use the |
| corresponding static library as input to make the |
| shared library, or the list of object files. |
| |
| obj2dynlib - function that produces build file lines to build a |
| dynamically loadable library file ("libfoo.so" on |
| Unix) from object files. |
| |
| called like this: |
| |
| obj2dynlib(lib => "PATH/TO/libfile", |
| objs => [ "PATH/TO/objectfile", ... ], |
| deps => [ "PATH/TO/otherlibfile", |
| ... ]); |
| |
| This is almost the same as libobj2shlib, but the |
| intent is to build a shareable library that can be |
| loaded in runtime (a "plugin"...). The differences |
| are subtle, one of the most visible ones is that the |
| resulting shareable library is produced from object |
| files only. |
| |
| obj2bin - function that produces build file lines to build an |
| executable file from object files. |
| |
| called like this: |
| |
| obj2bin(bin => "PATH/TO/binfile", |
| objs => [ "PATH/TO/objectfile", ... ], |
| deps => [ "PATH/TO/libfile", ... ]); |
| |
| 'bin' has the intended executable file name |
| *without* extension, obj2bin is expected to add |
| that. 'objs' has the list of object files (also |
| *without* extension) to build this library. 'deps' |
| has the list of library files (also *without* |
| extension) that the programs needs to be linked |
| with. |
| |
| in2script - function that produces build file lines to build a |
| script file from some input. |
| |
| called like this: |
| |
| in2script(script => "PATH/TO/scriptfile", |
| sources => [ "PATH/TO/infile", ... ]); |
| |
| 'script' has the intended script file name. |
| 'sources' has the list of source files to build the |
| resulting script from. |
| |
| Along with the build-file templates is the driving engine |
| Configurations/common.tmpl, which looks through all the information in |
| %unified_info and generates all the rulesets to build libraries, |
| programs and all intermediate files, using the rule generating |
| functions defined in the build-file template. |
| |
| As an example with the smaller build.info set we've seen as an |
| example, producing the rules to build 'libcrypto' would result in the |
| following calls: |
| |
| # Note: libobj2shlib will only be called if shared libraries are |
| # to be produced. |
| # Note 2: libobj2shlib gets both the name of the static library |
| # and the names of all the object files that go into it. It's up |
| # to the implementation to decide which to use as input. |
| # Note 3: common.tmpl peals off the ".o" extension from all object |
| # files, as the platform at hand may have a different one. |
| libobj2shlib(shlib => "libcrypto", |
| lib => "libcrypto", |
| objs => [ "crypto/aes", "crypto/evp", "crypto/cversion" ], |
| deps => [ ]); |
| |
| obj2lib(lib => "libcrypto" |
| objs => [ "crypto/aes", "crypto/evp", "crypto/cversion" ]); |
| |
| src2obj(obj => "crypto/aes" |
| srcs => [ "crypto/aes.c" ], |
| deps => [ ], |
| incs => [ "include" ], |
| intent => "lib"); |
| |
| src2obj(obj => "crypto/evp" |
| srcs => [ "crypto/evp.c" ], |
| deps => [ ], |
| incs => [ "include" ], |
| intent => "lib"); |
| |
| src2obj(obj => "crypto/cversion" |
| srcs => [ "crypto/cversion.c" ], |
| deps => [ "crypto/buildinf.h" ], |
| incs => [ "include" ], |
| intent => "lib"); |
| |
| generatesrc(src => "crypto/buildinf.h", |
| generator => [ "util/mkbuildinf.pl", "\"$(CC)", |
| "$(CFLAGS)\"", "\"$(PLATFORM)\"" ], |
| generator_incs => [ "util" ], |
| generator_deps => [ "util/Foo.pm" ], |
| incs => [ ], |
| deps => [ ], |
| intent => "lib"); |
| |
| The returned strings from all those calls are then concatenated |
| together and written to the resulting build-file. |