有什么好的方法来组织输入文件(Makefiles,SConstruct,CMakeLists.txt等)以构建自动化软件?


13

我喜欢对我的代码做的一件事是确保将其重构为可管理的部分。但是,在构建软件时,我发现无论我最终使用的构建自动化软件(最近是GNU Make或SCons)最终都变得一团糟。输入文件看起来像长脚本,似乎不容易重构。我希望能够以某种方式对其进行重构,但是“功能”的概念在某些构建自动化软件中的行为与编程语言中的行为并不完全相同,因此我发现很难编写可管理的代码任何中等复杂项目的Makefile文件或SConscript文件。

在编写用于构建自动化软件的可管理输入文件方面,有任何建议吗?与软件无关的建议是最好的,但是即使是关于特定构建自动化工具的建议也将有所帮助,尤其是Make或SCons,因为这就是我在项目中一直使用的。

编辑:正如Thorbjørn所指出的,我应该添加一些上下文和用法示例。我正在完成化学工程博士学位,并从事计算科学方面的研究。(我是SciComp.SE上的一名程序员,对于那些访问的人来说。)我的项目通常涉及混合编译语言(C,C ++,Fortran),它们执行某些繁重的脚本语言(Python) ,Perl)用于原型制作,偶尔也可以使用特定领域的语言进行原型制作或技术用途。

我在下面添加了两个示例,大致在250行范围内。对我来说,问题通常是缺乏模块化。这些项目中的一些可以组织成模块化的单元,并且最好按照这些思路抽象出构建的各个部分,以使我和将来的维护者更容易跟踪。将每个脚本分解成多个文件一直是我脑子里一直在想的一个解决方案。

第二个示例特别重要,因为我很快将不得不处理大量文件。

这是一条265行Makefile对我来说的样子,取自一个实际项目,并尽我所能组织:

#!/usr/bin/make
#Directory containing DAEPACK library folder
daepack_root = .
library = $(daepack_root)/lib
wrappers = $(daepack_root)/Wrappers/DSL48S
c_headers = parser.h problemSizes.h
f77_headers=problemSizes.f commonParam.f
f90_headers=problemSizes.f commonParam.f90
includes = -I. -Iinclude -I/usr/include/glib-2.0 \
    -I/usr/lib/glib-2.0/include -I/usr/include/libxml2 \
    -I/usr/include/libgdome -I/usr/include/gtest/

#Fortran 77 environment variables
f77=gfortran
fflags=-ggdb -cpp -fno-second-underscore --coverage -falign-commons \
    -mcmodel=large -fbacktrace -pg 
flibs=

#Fortran 90 environment variables
f90=gfortran
f90flags=-ggdb -cpp -fno-second-underscore --coverage -falign-commons \
    -mcmodel=large -fbacktrace -pg 
f90libs=

#C environment flags
cc=gcc
cflags=-ggdb --coverage $(includes) -mcmodel=large 
clibs=

#Libraries for linking
libs=-L$(library) -ldaepack_sparse -lblas -llapack -ldl -lg2c \
    -lgdome -lxml2 -lgtest -lcunit -lcholmod -lamd -lcolamd -lccolamd \
    -lmetis -lspqr -lm -lblas -llapack -lstdc++ -lpcre

#Object files
objs=main.o $(dsl48sObjs) $(gdxObjs)
gdxObjs = gdxf9def.o gdxf9glu.o gamsglobals_mod.o 
commonObjs=libdsl48s_model.sl cklib.o parser.o $(gdxObjs)
originalModelObjs=originalModel.o dsl48sChemkinModule.o $(commonObjs)
cspSlowModelObjs=cspSlowModel.o dsl48sChemkinModuleSlow.o cspModule.o \
    $(commonObjs)
orthoProjModelObjs=orthoProjModel.o dsl48sChemkinModuleOrthoProj.o \
    orthoProjModule.o basisModule.o spqrUtility.o $(commonObjs)

#Shell environment variable definitions for FUnit
FCFLAGS := $(f90flags)
LDFLAGS := libdsl48s_model.sl cklib.o gdxf9glu.o parser.o spqrUtility.o \
    $(libs)

misc=*table *size.f 
output=*.out

#Ftncheck flags for static analysis of Fortran 77 code
ftnchekflags= -declare -include=. -library -style=block-if,distinct-do,do-enddo,end-name,goto,labeled-stmt,structured-end

all: ckinterp.exe parserTest.exe originalModel.exe cspSlowModel.exe \
    orthoProjModel.exe spqrUtilityTest.exe
#Check code style with lexical analyzer
    @echo Checking program style...
    ftnchek $(ftnchekflags) rhs.f
    ftnchek $(ftnchekflags) resorig.f
    ftnchek $(ftnchekflags) res.f
#   ftnchek $(ftnchekflags) cklib.f
#   ftnchek $(ftnchekflags) ckinterp.f
#Set up baseline coverage data file
    @echo Set up baseline coverage data file
    lcov -c -i -d . -o conpDSL48Sbase.info
#Run unit test on cspModule.f90
    @echo Running unit tests on cspModule.f90...
    funit cspModule
#Generate test coverage data for cspModule.f90
    @echo Generating test coverage data from cspModule.f90 tests...
    lcov -c -d . -o conpDSL48ScspTest.info
#Run unit test on orthoProjModule.f90
    @echo Running unit tests on orthoProjModule.f90...
    funit orthoProjModule
#Generate test coverage data for orthoProjModule.f90
    @echo Generating test coverage data from orthoProjModule.f90 tests...
    lcov -c -d . -o conpDSL48SgenProjTest.info
#Run unit tests on the parser C library
    @echo Running unit tests on parser in C...
    -G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind -v --tool=memcheck \
    --leak-check=full --show-reachable=yes --leak-resolution=high \
    --num-callers=20 --log-file=parserTest.vgdump \
    ./parserTest.exe > parserTest.log
#Generate test coverage data for the parser wrapper C library
    @echo Generating test coverage data for the parser in C...
    lcov -c -d . -o conpDSL48SparserTest.info
#Run unit tests on the SparseQR C library
    @echo Running unit tests on SparseQR library in C...
    ./spqrUtilityTest.exe
#Generate test coverage data for the SparseQR C library
    @echo Generating test coverage data for the SparseQR C library...
    lcov -c -d . -o conpDSL48SsparseTest.info
#Run unit test on basisModule.f90
    @echo Running unit tests on basisModule.f90...
    funit basisModule
#Generate test coverage data for basisModule.f90
    @echo Generating test coverage data from basisModule.f90 tests...
    lcov -c -d . -o conpDSL48SbasisMod.info
#Combine test coverage data
    @echo Combine baseline and test coverage data...
    lcov -a conpDSL48Sbase.info \
    -a conpDSL48ScspTest.info \
    -a conpDSL48SgenProjTest.info \
    -a conpDSL48SbasisMod.info \
    -a conpDSL48SparserTest.info \
    -a conpDSL48SsparseTest.info \
    -o conpDSL48Stotal.info
#Post-process to remove coverage statistics from automatically 
#generated source code.
    @echo Removing coverage statistics for automatically generated source...
    lcov -r conpDSL48Stotal.info basisModule_fun.f90 \
    ckinterp.f cklib.f cspModule_fun.f90 davisSkodjeAd.f90 \
    davisSkodjeJac.f90 davisSkodjeRes.f90 davisSkodjeRhs.f90 \
    davisSkodjeSp.f90 gdxf9def.f90 gdxf9glu.c orthoProjModule_fun.f90 \
    jac.f jacorig.f resad.f resadp.f resorigad.f resorigadp.f ressp.f \
    resorigsp.f senrhs.f senrhsorig.f TestRunner.f90 \
    -o conpDSL48Stotal.info
#Generate HTML report of coverage data
    @echo Generate HTML report of coverage data...
    genhtml conpDSL48Stotal.info
    @echo Open "index.html" in browser for coverage results!

originalModel.exe: $(originalModelObjs) $(f90_headers) $(f77_headers) \
    $(c_headers)
    $(f90) $(f90flags) -o originalModel.exe $(originalModelObjs) $(libs)

originalModel.o: dsl48sChemkinModule.o $(commonObjs) $(f77_headers) \
    $(f90_headers) $(c_headers)
    $(f90) $(f90flags) -c -o originalModel.o originalModel.f90

cspSlowModel.exe: $(cspSlowModelObjs) $(f90_headers) $(f77_headers) \
    $(c_headers)
    $(f90) $(f90flags) -o cspSlowModel.exe $(cspSlowModelObjs) $(libs)

cspSlowModel.o: dsl48sChemkinModuleSlow.o cspModule.o $(commonObjs) \
    $(c_headers) $(f77_headers)
    $(f90) $(f90flags) -c -o cspSlowModel.o cspSlowModel.f90

orthoProjModel.exe: $(orthoProjModelObjs) $(f90_headers) $(f77_headers) \
    $(c_headers) resOrthoFast.o
    $(f90) $(f90flags) -o orthoProjModel.exe $(orthoProjModelObjs) \
    resOrthoFast.o $(libs)

orthoProjModel.o: dsl48sChemkinModuleOrthoProj.o orthoProjModule.o $(commonObjs) \
    $(c_headers) $(f90_headers) $(f77_headers) resOrthoFast.o basisModule.o
    $(f90) $(f90flags) -c -o orthoProjModel.o orthoProjModel.f90

dsl48sChemkinModule.o: dsl48sChemkinModule.f90 cklib.o problemSizes.h \
    parser.o $(c_headers) $(f90_headers)
    $(f90) $(f90flags) -c -o dsl48sChemkinModule.o dsl48sChemkinModule.f90 

dsl48sChemkinModuleSlow.o: dsl48sChemkinModuleSlow.f90 cspModule.o cklib.o \
    problemSizes.h parser.o $(c_headers) $(f90_headers)
    $(f90) $(f90flags) -c -o dsl48sChemkinModuleSlow.o \
    dsl48sChemkinModuleSlow.f90

dsl48sChemkinModuleOrthoProj.o: dsl48sChemkinModuleOrthoProj.f90 \
    orthoProjModule.o basisModule.o cklib.o problemSizes.h \
    parser.o $(c_headers) $(f90_headers)
    $(f90) $(f90flags) -c -o dsl48sChemkinModuleOrthoProj.o \
    dsl48sChemkinModuleOrthoProj.f90

basisModule.o: basisModule.f90 cklib.o spqrUtility.o commonParam.f90
    $(f90) $(f90flags) -c -o basisModule.o basisModule.f90

spqrUtility.o: spqrUtility.h spqrUtility.c
    $(cc) $(cflags) -c -o spqrUtility.o spqrUtility.c

spqrUtilityTest.exe: spqrUtilityTest.o spqrUtility.o
    $(cc) $(cflags) -o spqrUtilityTest.exe spqrUtilityTest.o \
    spqrUtility.o $(libs)

spqrUtilityTest.o: spqrUtilityTest.c spqrUtility.o
    $(cc) $(cflags) -c -o spqrUtilityTest.o spqrUtilityTest.c

cklib.o: cklib.f ckstrt.f
    $(f77) $(fflags) -c -o cklib.o cklib.f

ckinterp.exe: ckinterp.o
    $(f77) $(fflags) -o ckinterp.exe ckinterp.o

ckinterp.o: ckinterp.f
    $(f77) $(fflags) -c -o ckinterp.o ckinterp.f

#Recursive makefile inherited from previous graduate students
libdsl48s_model.sl: $(f77_headers) cklibDAEPACK.f
    cp $(wrappers)/makefile model.mk
    make -f model.mk

resOrthoFast.o: libdsl48s_model.sl
    $(f90) $(f90flags) -c -o resOrthoFast.o resOrthoFast.f90

problemSizes.f: problemSizes.fpp problemSizes.h
    cpp problemSizes.fpp problemSizes.f
    perl -p -i.bak -we 's/# /! /;' problemSizes.f

commonParam.f90: commonParam.f
    perl -p -i.bak -we 's/^#/!/;' commonParam.f
    echo "commonParam t f t fpp" | pref77tof90
    echo "commonParam /" | f77tof90
    perl -p -i.bak -we 's/integer a/!integer a/;' commonParam.f
    perl -p -i.bak -we 's/END   //;' commonParam.f90

commonParam.f: commonParam.fpp problemSizes.h
    cpp commonParam.fpp commonParam.f
    perl -p -i.bak -we 's/^#/!/;' commonParam.f

cspModule.o: cspModule.f90
    $(f90) $(f90flags) -c -o cspModule.o cspModule.f90

orthoProjModule.o: gamsglobals_mod.o gdxf9def.o gdxf9glu.o orthoProjModule.f90 \
    formatLabels.f90
    $(f90) $(f90flags) -c -o orthoProjModule.o orthoProjModule.f90

gdxf9def.o: gdxf9def.f90
    $(f90) $(f90flags) -c -o gdxf9def.o gdxf9def.f90

gdxf9glu.o: gdxf9glu.c gdxf9def.o
#64-bit version of wrappers (with underscores)
    $(cc) $(cflags) -DCIA_LEX -DAPIWRAP_LCASE_DECOR -c -o \
    gdxf9glu.o gdxf9glu.c
#64-bit version of wrappers (without underscores, for C interoperability)
#   $(cc) $(cflags) -DCIA_LEX -DAPIWRAP_LCASE_NODECOR -c gdxf9glu.c
#32-bit version of wrappers
#   $(cc) $(cflags) -DAPIWRAP_LCASE_DECOR -c gdxf9glu.c -Iinclude

gamsglobals_mod.o: gamsglobals_mod.f90 gdxf9def.o gdxf9glu.o
    $(f90) $(f90flags) -c gamsglobals_mod.f90

parser.o: parser.c $(c_headers)
    $(cc) $(cflags) -c -o parser.o parser.c 

parserTest.exe: parserTest.o parser.o
    $(cc) $(cflags) -o parserTest.exe parser.o \
    parserTest.o $(libs)

parserTest.o: parserTest.cpp parser.o
    $(cc) $(cflags) -c -o parserTest.o parserTest.cpp

clean:
    -rm *.bak
    -rm *.f77
    -rm *.log
    -rm commonParam.f90
    -rm problemSizes.f
    -rm commonParam.f
    -make clean -f model.mk
    -rm model.mk
    -rm *.o
    -rm *.mod
    -rm $(misc)
    -rm *.exe
    -funit --clean
    -rm *.gcno
    -rm *.gcda
    -rm *.info
    -rm *.png
    -rm *.html
    -rm *.css
    -rm -rf html
    -rm *.pyc
    -rm *.lst

这是SConstruct我目前正在尝试为一个大致复杂的项目组织的245行文件:

## \file SConstruct
#  \brief Compiles the library and compiles tests.
#

import SCons

## \brief Build up directory names of each COIN library from package names
#         and versions.
#

## Overall SCons environment
#
env = Environment();

flags = []

## Compile using debug versions?
#
debug = True
debugString = '-debug'
debugFlags = ['-ggdb']

dynamicLinkFlag = '-Wl,-rpath,'

if debug:
    flags += debugFlags

## Compile Google Test from scratch.
#
GTestVersion = '1.6.0'
GTestStem = 'gtest-' + GTestVersion
GTestBuildIncDir = [GTestStem,
                    GTestStem + '/include',
                    ]
GTestAllLib = env.Library('lib/libgtest.a', 'gtest-1.6.0/src/gtest-all.cc',
                      CPPPATH = GTestBuildIncDir,
                      CXXFLAGS = flags)
GTestMainLib = env.Library('lib/libgtest_main.a',
                           'gtest-1.6.0/src/gtest_main.cc',
                           CPPPATH = GTestBuildIncDir,
                           CXXFLAGS = flags)

GTestIncDir = GTestStem + '/include/gtest'
GTestLibDir = 'lib'
GTestLibFlags = ['gtest', 'gtest_main', 'pthread']

## Armadillo matrix library
#
ArmadilloLibFlags = ['armadillo'];

## Quick reminder of SCons flags:
#  CPPPATH = path of headers (include directories)
#  LIBPATH = path of libraries
#  LIBS = flags of libraries
#  CXXFLAGS = C++ compilation flags
#




## Locations of libraries installed on system in standard locations
#
StdIncDir = '/usr/include'
StdLibDir = '/usr/lib'

## Configuration information for COIN libraries
#
CoinUtilsVersion = '2.6.4'
ClpVersion = '1.12.0'
OsiVersion = '0.103.0'
CbcVersion = '2.5.0'

## Some standard directory locations of COIN libraries, with slashes added for
#  for convenience.
#
CoinLibLocation = '/usr/local/COIN/'
StdCoinIncDir = '/include/coin'
StdCoinLibDir = '/lib'

CoinUtilsStem = 'CoinUtils-' + CoinUtilsVersion
ClpStem = 'Clp-' + ClpVersion
OsiStem = 'Osi-' + OsiVersion
CbcStem = 'Cbc-' + CbcVersion

if debug:
    CoinUtilsStem += debugString
    CbcStem += debugString
    ClpStem += debugString
    OsiStem += debugString


## Build up include directory names for COIN projects from constituent parts.
#
CoinUtilsIncDir = CoinLibLocation + CoinUtilsStem + StdCoinIncDir
ClpIncDir = CoinLibLocation + ClpStem + StdCoinIncDir
OsiIncDir = CoinLibLocation + OsiStem + StdCoinIncDir
CbcIncDir = CoinLibLocation + CbcStem + StdCoinIncDir

## Build up library names from COIN projects from constituent parts
#
CoinUtilsLibDir = CoinLibLocation + CoinUtilsStem + StdCoinLibDir
ClpLibDir = CoinLibLocation + ClpStem + StdCoinLibDir
OsiLibDir = CoinLibLocation + OsiStem + StdCoinLibDir
CbcLibDir = CoinLibLocation + CbcStem + StdCoinLibDir

## CPLEX
#
CpxStem = '/opt/ibm/ILOG/CPLEX_Studio_Academic123/cplex/'
CpxIncDir = CpxStem + 'include/ilcplex'
CpxLibDir = CpxStem + 'lib/x86-64_sles10_4.1/static_pic'

## Gurobi
# 
GrbStem = '/opt/gurobi460/linux64/'
GrbIncDir = GrbStem + 'include'
GrbLibDir = GrbStem + 'lib'

OsiLibFlags = ['Osi', 'CoinUtils']
ClpLibFlags = ['Clp', 'OsiClp']
CbcLibFlags = ['Cbc', 'Cgl']
OsiCpxLibFlags = ['OsiCpx']
OsiGrbLibFlags = ['OsiGrb']
CpxLibFlags = ['cplex', 'ilocplex', 'pthread', 'm']
GrbLibFlags = ['gurobi_c++', 'gurobi46', 'pthread', 'm']

milpIncDirs = [CoinUtilsIncDir,
               ClpIncDir,
               OsiIncDir,
               CbcIncDir,
               CpxIncDir,
               GrbIncDir,
            GTestIncDir,
               ]
milpLibDirs = [CoinUtilsLibDir,
               ClpLibDir,
               OsiLibDir,
               CbcLibDir,
               CpxLibDir,
               GrbLibDir,
               GTestLibDir,
            ]
milpLibFlags = [OsiCpxLibFlags,
                OsiGrbLibFlags,
                CbcLibFlags,
                ClpLibFlags,
                OsiLibFlags,
                CpxLibFlags,
                GrbLibFlags,
                GTestLibFlags,
                ]
##milpSolver = env.Object('milpSolver.cpp',
            ##                         CPPPATH = milpIncDirs,
##                         LIBPATH = milpLibDirs,
##                         CXXFLAGS = flags)
milpSolverTest = env.Program('milpSolverUnitTest',
                                  ['milpSolverTest.cpp',
                                   'milpSolver.cpp'],
                                  CPPPATH = milpIncDirs,
                                  LIBPATH = milpLibDirs,
                                  LIBS = milpLibFlags,
                                  CXXFLAGS = flags,
                                  LINKFLAGS = ['-Wl,-rpath,' + OsiLibDir])
env.Depends(milpSolverTest, [GTestAllLib, GTestMainLib])

## Chemkin source directories and files
#
ChemkinSourceDir = '/mnt/hgfs/DataFromOldLaptop/Data/ModelReductionResearch/Papers/AdaptiveChemistryPaper/AdaptiveChemistry/NonOpenSource/ChemkinII/';
ChemkinSourceList = ['cklib.f', 'pcmach.f','tranlib.f']
ChemkinSourceList = [ChemkinSourceDir + FileName
                     for FileName in ChemkinSourceList]
env.Depends('cklib.f','ckstrt.f')

## Cantera include directorie
#
CanteraStem = '/usr/local/cantera'

if debug:
    CanteraStem += debugString

CanteraIncDir = CanteraStem + '/include/cantera'
CanteraLibDir = CanteraStem + '/lib'
CanteraTestingFlags = ['kinetics', 'thermo', 'tpx', 'ctbase', 'm',]
CanteraLibFlags = ['user', 'oneD', 'zeroD', 'equil', 'kinetics', 'transport',
                    'thermo', 'ctnumerics', 'ctmath', 'tpx', 'ctspectra',
                    'converters', 'ctbase', 'cvode', 'ctlapack', 'ctblas',
                    'ctf2c', 'ctcxx', 'ctf2c', 'm', 'm', 'stdc++']

CxxFortranFlags = ['g2c', 'gfortran']; 

chemSolverIncDir = [CanteraIncDir,
                    StdIncDir,
                    '/usr/local/include',
                    GTestIncDir,
                    ]
chemSolverLibDir = [StdLibDir,
                    CanteraLibDir,
                    GTestLibDir,
                    ]
chemSolverLibFlags = [GTestLibFlags,
                      CxxFortranFlags,
                      CanteraLibFlags,
                      ArmadilloLibFlags,
                      ]

chemSolverTest = env.Program('chemSolverUnitTest',
                        ['chemSolverTest.cpp',
                         'chemSolver.cpp',
                         'ckwrapper.f90'] + ChemkinSourceList,
                        CPPPATH = chemSolverIncDir,
                        LIBPATH = chemSolverLibDir,
                        LIBS = chemSolverLibFlags,
                        CXXFLAGS = flags,
                        FORTRANFLAGS = flags,
                        F90FLAGS = flags)
env.Depends(chemSolverTest, [GTestAllLib, GTestMainLib])

#env.AddPostAction(milpSolverTest, milpSolverTest[0].abspath)
testAlias = env.Alias('test', [milpSolverTest, chemSolverTest])
AlwaysBuild(testAlias)

ckInterp = env.Program('ckinterp', ChemkinSourceDir + 'ckinterp.f')

canteraGTestLibFlags = CanteraTestingFlags + GTestLibFlags

#canteraGTestLibFlags = ['kinetics', 'thermo', 'tpx',
#                        'ctbase',  'm', 'gtest', 'gtest_main', 'pthread']

canteraGTest = env.Program('canteraGTest',
                           'canteraGTest.cpp',
                           CPPPATH = chemSolverIncDir,
                           LIBPATH = chemSolverLibDir,
                           LIBS = canteraGTestLibFlags,
                           CXXFLAGS = flags)
env.Depends(canteraGTest, [GTestAllLib, GTestMainLib])

canteraMemTestLibFlags = CanteraTestingFlags

canteraMemTest = env.Program('canteraMemTest',
                             'canteraMemTest.cpp',
                             CPPPATH = chemSolverIncDir,
                             LIBPATH = chemSolverLibDir,
                             LIBS = canteraMemTestLibFlags,
                             CXXFLAGS = flags)

Answers:


8

重构策略

除了Dietbuddha和ThorbjørnRavnAnderson的评论之外,重构构建脚本的另一种方法是将它们分成多个文件。您如何执行此操作取决于构建系统。

对于Make,它就像使用include命令一样简单,如“认为递归使有害”中所建议的那样。此伪指令的工作方式#include与C预处理器中的工作方式相同,并处理包含的文件,就像已剪切并粘贴该文件来代替include命令一样。使用该include命令,可以Makefile通过将模块化零件移入子零件来重构主零件Makefile

CMake也有类似的命令。

SCons需要使用不同命令的类似方法。将构建脚本分为一个主脚本和几个较小的子脚本的基本思想保持不变,但是SCons不会将较小的脚本的文本直接包含在主构建脚本中,而是将较小的脚本像单独的命名空间一样对待(因为SCons使用Python代替Shell)。SCons Environment对象具有一种称为的方法SConscript(),使您可以将对象从SConstruct文件导入到称为SConscript文件的子文件中,该子文件可用于重构构建脚本。的基本思想SConscript文件和SConscript()命令,可以发现这里SCons的维基SConscripts可以找到如何在分层构建中使用的示例这里的在SCons的用户指南

从这三个示例推断,似乎一般的策略是通过将构建脚本拆分为一个调用多个文件的主脚本来重构构建脚本。对于使用的特定构建自动化软件,这是怎么做的。

重温SCons示例

SConstruct上面的文件中,我将所有配置信息移至名为的模块中build_config.py。所有文字都存在于全局名称空间中,这对于Python可能是危险的,但是很容易(尽管有些乏味)可修复。我进行了检查,以确保没有名称冲突__builtin__(Python内置命名空间,因此我不会覆盖任何重要的对象)。

## \file build_config.py
#  \brief Sets configuration of file locations manually.
#

## Flags for compilers
#

flags = []

## Compile using debug versions?
#
debug = True
debugString = '-debug'
debugFlags = ['-ggdb']

dynamicLinkFlag = '-Wl,-rpath,'

if debug:
    flags += debugFlags

## Configuration information for GTest
#
GTestVersion = '1.6.0'
GTestStem = 'gtest-' + GTestVersion
GTestBuildIncDir = [GTestStem,
                    GTestStem + '/include',
                    ]

GTestIncDir = GTestStem + '/include/gtest'
GTestLibDir = 'lib'
GTestLibFlags = ['gtest', 'gtest_main', 'pthread']

## Configuration information for Armadillo matrix library
#

ArmadilloLibFlags = ['armadillo'];

## Locations of libraries installed on system in standard locations
#
StdIncDir = '/usr/include'
StdLibDir = '/usr/lib'

## Configuration information for COIN libraries
#
CoinUtilsVersion = '2.6.4'
ClpVersion = '1.12.0'
OsiVersion = '0.103.0'
CbcVersion = '2.5.0'

## Standard directory locations of COIN libraries, with slashes added for
#  for convenience.
#
CoinLibLocation = '/usr/local/COIN/'
StdCoinIncDir = '/include/coin'
StdCoinLibDir = '/lib'

CoinUtilsStem = 'CoinUtils-' + CoinUtilsVersion
ClpStem = 'Clp-' + ClpVersion
OsiStem = 'Osi-' + OsiVersion
CbcStem = 'Cbc-' + CbcVersion

if debug:
    CoinUtilsStem += debugString
    CbcStem += debugString
    ClpStem += debugString
    OsiStem += debugString

## Build up include directory names for COIN projects from constituent parts.
#
CoinUtilsIncDir = CoinLibLocation + CoinUtilsStem + StdCoinIncDir
ClpIncDir = CoinLibLocation + ClpStem + StdCoinIncDir
OsiIncDir = CoinLibLocation + OsiStem + StdCoinIncDir
CbcIncDir = CoinLibLocation + CbcStem + StdCoinIncDir

## Build up library names from COIN projects from constituent parts
#
CoinUtilsLibDir = CoinLibLocation + CoinUtilsStem + StdCoinLibDir
ClpLibDir = CoinLibLocation + ClpStem + StdCoinLibDir
OsiLibDir = CoinLibLocation + OsiStem + StdCoinLibDir
CbcLibDir = CoinLibLocation + CbcStem + StdCoinLibDir

## CPLEX
#
CpxStem = '/opt/ibm/ILOG/CPLEX_Studio_Academic123/cplex/'
CpxIncDir = CpxStem + 'include/ilcplex'
CpxLibDir = CpxStem + 'lib/x86-64_sles10_4.1/static_pic'

## Gurobi
# 
GrbStem = '/opt/gurobi460/linux64/'
GrbIncDir = GrbStem + 'include'
GrbLibDir = GrbStem + 'lib'

OsiLibFlags = ['Osi', 'CoinUtils']
ClpLibFlags = ['Clp', 'OsiClp']
CbcLibFlags = ['Cbc', 'Cgl']
OsiCpxLibFlags = ['OsiCpx']
OsiGrbLibFlags = ['OsiGrb']
CpxLibFlags = ['cplex', 'ilocplex', 'pthread', 'm']
GrbLibFlags = ['gurobi_c++', 'gurobi46', 'pthread', 'm']

milpIncDirs = [CoinUtilsIncDir,
               ClpIncDir,
               OsiIncDir,
               CbcIncDir,
               CpxIncDir,
               GrbIncDir,
            GTestIncDir,
               ]
milpLibDirs = [CoinUtilsLibDir,
               ClpLibDir,
               OsiLibDir,
               CbcLibDir,
               CpxLibDir,
               GrbLibDir,
               GTestLibDir,
            ]
milpLibFlags = [OsiCpxLibFlags,
                OsiGrbLibFlags,
                CbcLibFlags,
                ClpLibFlags,
                OsiLibFlags,
                CpxLibFlags,
                GrbLibFlags,
                GTestLibFlags,
                ]

## Configuration information for Chemkin source directories and files
#
ChemkinSourceDir = '/mnt/hgfs/DataFromOldLaptop/Data/ModelReductionResearch/Papers/AdaptiveChemistryPaper/AdaptiveChemistry/NonOpenSource/ChemkinII/';
ChemkinSourceList = ['cklib.f', 'pcmach.f','tranlib.f']
ChemkinSourceList = [ChemkinSourceDir + FileName
                     for FileName in ChemkinSourceList]

## Configuration information for Cantera
#
CanteraStem = '/usr/local/cantera'

if debug:
    CanteraStem += debugString

CanteraIncDir = CanteraStem + '/include/cantera'
CanteraLibDir = CanteraStem + '/lib'
CanteraTestingFlags = ['kinetics', 'thermo', 'tpx', 'ctbase', 'm',]
CanteraLibFlags = ['user', 'oneD', 'zeroD', 'equil', 'kinetics', 'transport',
                    'thermo', 'ctnumerics', 'ctmath', 'tpx', 'ctspectra',
                    'converters', 'ctbase', 'cvode', 'ctlapack', 'ctblas',
                    'ctf2c', 'ctcxx', 'ctf2c', 'm', 'm', 'stdc++']

CxxFortranFlags = ['g2c', 'gfortran']; 

chemSolverIncDir = [CanteraIncDir,
                    StdIncDir,
                    '/usr/local/include',
                    GTestIncDir,
                    ]
chemSolverLibDir = [StdLibDir,
                    CanteraLibDir,
                    GTestLibDir,
                    ]
chemSolverLibFlags = [GTestLibFlags,
                      CxxFortranFlags,
                      CanteraLibFlags,
                      ArmadilloLibFlags,
                      ]
canteraGTestLibFlags = CanteraTestingFlags + GTestLibFlags

#canteraGTestLibFlags = ['kinetics', 'thermo', 'tpx',
#                        'ctbase',  'm', 'gtest', 'gtest_main', 'pthread']

SConstruct文件调用一堆SConscript文件来构建包含我代码中具有的主要功能的模块。将大多数构建命令移至SConscript文件可以使SConstruct文件非常简单:

## \file SConstruct
#  \brief Compiles the library and compiles tests.
#

import SCons
from build_config import *

## \brief Build up directory names of each COIN library from package names
#         and versions.
#

## Overall SCons environment
#
env = Environment();

## Compile Google Test from source using SConscript file.
#
GTestAllLib, GTestMainLib = env.SConscript('gtest.scons',
                                           exports=['env'])

## Compile MILP solver module and tests from source using SConscript file.
#
milpSolverTest = env.SConscript('milpSolver.scons',
                                exports=['env'])

## Compile chemistry solver module and associated tests from source
#  using SConscript file.
chemSolverTest, canteraGTest = env.SConscript('chemSolver.scons',
                                              exports=['env'])

## Since all tests use GTest, make the dependency of the module
# tests on the GTest libraries explicit.
env.Depends(milpSolverTest, [GTestAllLib, GTestMainLib])
env.Depends(chemSolverTest, [GTestAllLib, GTestMainLib])
env.Depends(canteraGTest, [GTestAllLib, GTestMainLib])

#env.AddPostAction(milpSolverTest, milpSolverTest[0].abspath)
testAlias = env.Alias('test', [milpSolverTest, chemSolverTest])
AlwaysBuild(testAlias)

然后,所有三个SConscript文件都具有扩展名.scons,并将项目分解为代表不同功能的模块。

Google测试SConscript文件:

## \file gtest.scons
#  \brief SConscript file that contains information for SCons build of
#  GTest. Use Python syntax highlighting for source.

from build_config import *
Import('env')

## Compile Google Test from scratch.
#
GTestAllLib = env.Library('lib/libgtest.a', 'gtest-1.6.0/src/gtest-all.cc',
                      CPPPATH = GTestBuildIncDir,
                      CXXFLAGS = flags)
GTestMainLib = env.Library('lib/libgtest_main.a',
                           'gtest-1.6.0/src/gtest_main.cc',
                           CPPPATH = GTestBuildIncDir,
                           CXXFLAGS = flags)

Return('GTestAllLib', 'GTestMainLib')

混合整数线性规划求解器SConscript文件:

## \file milpSolver.scons
#  \brief SConscript file that contains information for SCons build of
#  mixed-integer linear programming solver module. Use Python syntax
#  highlighting for source.

from build_config import *
Import('env')

## Compile MILP solver module and tests.
#

##milpSolver = env.Object('milpSolver.cpp',
            ##                         CPPPATH = milpIncDirs,
##                         LIBPATH = milpLibDirs,
##                         CXXFLAGS = flags)
milpSolverTest = env.Program('milpSolverUnitTest',
                                  ['milpSolverTest.cpp',
                                   'milpSolver.cpp'],
                                  CPPPATH = milpIncDirs,
                                  LIBPATH = milpLibDirs,
                                  LIBS = milpLibFlags,
                                  CXXFLAGS = flags,
                                  LINKFLAGS = ['-Wl,-rpath,' + OsiLibDir])

Return('milpSolverTest')

化学引擎SConscript文件:

## \file chemSolver.scons
#  \brief  SConscript file that sets up SCons build of chemistry solver module.
#  Use Python syntax highlighting for source.

from build_config import *
Import('env')

## Compile CHEMKIN interpreter.
#
ckInterp = env.Program('ckinterp', ChemkinSourceDir + 'ckinterp.f')

## Enforce explicit dependence of CHEMKIN library on CHEMKIN
#  parameter file 'ckstrt.f' because SCons' scanner won't pick it up.
env.Depends('cklib.f','ckstrt.f')

chemSolverTest = env.Program('chemSolverUnitTest',
                        ['chemSolverTest.cpp',
                         'chemSolver.cpp',
                         'ckwrapper.f90'] + ChemkinSourceList,
                        CPPPATH = chemSolverIncDir,
                        LIBPATH = chemSolverLibDir,
                        LIBS = chemSolverLibFlags,
                        CXXFLAGS = flags,
                        FORTRANFLAGS = flags,
                        F90FLAGS = flags)

canteraGTest = env.Program('canteraGTest',
                           'canteraGTest.cpp',
                           CPPPATH = chemSolverIncDir,
                           LIBPATH = chemSolverLibDir,
                           LIBS = canteraGTestLibFlags,
                           CXXFLAGS = flags)

canteraMemTestLibFlags = CanteraTestingFlags

canteraMemTest = env.Program('canteraMemTest',
                             'canteraMemTest.cpp',
                             CPPPATH = chemSolverIncDir,
                             LIBPATH = chemSolverLibDir,
                             LIBS = canteraMemTestLibFlags,
                             CXXFLAGS = flags)

Return('chemSolverTest', 'canteraGTest')

这五个文件的组合可能比原始的长文件长一些,但是对我来说更容易管理,因为我可以将构建分离成一个配置文件和一堆大部分未连接的单元,因此我不必一次在我的脑海中跟踪整个构建。


4

它们是功能,它们遵循略有不同的规则。“功能”通常是构建的目标和规则。考虑它的另一种方法是构建DAG中的节点和线。“目标”是节点或构建工件,线条是有关如何转换先前节点的规则。

我首先通过弄清楚DAG是什么来重构构建脚本。然后隔离通用规则并进行重复数据删除。

这是一个简单的例子:

all:
   mkdir bigfiles
   cat file1 file2 > bigfiles/bigfile1
   cat file3 file4 > bigfiles/bigfile2

DAG应该是什么:

file1 \
        =-> bigfile1 \
file2 /               \
                        =-> all
file3 \               /
        =-> bigfile2 /
file4 /

新规则:

 bigfiles:
   mkdir bigfiles

 bigfiles/bigfile1: bigfiles
   cat file1 file2 > bigfiles/bigfile1

 bigfiles/bigfiles2: bigfiles
   cat file3 file4 > bigfiles/bigfile2

重复数据删除:

 BIGFILES:=bigfiles/bigfile1 bigfiles/bigfile2
 bigfiles/bigfile1:=file1 file2
 bigfiles/bigfile2:=file3 file4

 .PHONY: all
 all: $(BIGFILES)

 bigfiles:
    mkdir bigfiles

 $(BIGFILES): bigfiles
    cat $($@) > $@

在本练习结束时,我目前的代码比开始时要多。但是,您还有一个更通用的“功能”。经过适当参数设置的“功能”,因此因此更易于维护和扩展。


+1是关于查看DAG的想法。我同意那里有重构的潜在机会。可视化DAG可能会让人感到痛苦。对于Makefile中,有Makefile文件:: GraphViz的,而对于使用SCons,有一个Python脚本在这里,可能需要一点黑客,这取决于项目。脚本给我的图形有时非常大,因此我发现必须仔细过滤要可视化的内容。
Geoff Oxberry'2

您无需查看整个DAG。您可以只浏览SConstruct文件的Makefile并定位最差的部分。确定他们在做什么,以对隐式DAG进行返工并将其分解。将一般规则移到单独的文件中。就像使用常规编程语言一样,根据数据/功能/目标进行具有高度凝聚力的组织。
Dietbuddha'2

是的,当我过滤掉很多文件时,这就是我要尝试做的事情,因为SConstruct依赖解析器会拾取大量文件。因此,我将它们过滤掉,并在链接处使用Python脚本制作了一个点文件,以便可以直观地看到DAG中我认为重要的部分。它起作用了,并且我得到了一个足够小的图表,所以我可以挑选出重要的东西,但是我看不到有什么可以用的。我怀疑这种策略在Make中使用起来容易得多,因为在Makefile中查找重复的规则更容易,正如您从我发布的源中看到的那样。
Geoff Oxberry'2

2

通过将逻辑移到单独的脚本中,使GNU autoconf可以使makefile可维护和可移植。

我会考虑使用该步骤来管理Makefile。


我同意GNU Autoconf是为构建的可移植性和可维护性而设计的。有一次,我调查了学习Autoconf的过程,发现花了比我愿意投资的时间更多的时间。(我正在完成博士学位,所以构建此软件需要早于而不是晚进行。)也许在将来?对于我来说,可移植性甚至不是主要问题,这是构建脚本的长度和缺乏模块化,并且我不确定移植到Autoconf是否可以解决这些问题。
Geoff Oxberry'2

然后考虑显示一些实际问题以及为什么当前的解决方案不够好。同样,这在学术界也很重要。

我添加了一些示例来说明我现在正在使用的工具。
Geoff Oxberry 2012年
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.