因此,我有两个YAML文件“ A”和“ B”,我希望将A的内容插入B内,或者拼接到现有的数据结构(如数组)中,或者作为元素的子级(如值)对于某个哈希键。
这有可能吗?怎么样?如果不是,是否有任何指向规范性引用的指针?
因此,我有两个YAML文件“ A”和“ B”,我希望将A的内容插入B内,或者拼接到现有的数据结构(如数组)中,或者作为元素的子级(如值)对于某个哈希键。
这有可能吗?怎么样?如果不是,是否有任何指向规范性引用的指针?
Answers:
不,YAML不包含任何类型的“导入”或“包含”语句。
您的问题并不要求提供Python解决方案,但这是一个使用PyYAML的解决方案。
PyYAML允许您将自定义构造函数(例如!include
)附加到YAML加载程序。我提供了一个可以设置的根目录,以便该解决方案支持相对和绝对文件引用。
这是一个基于类的解决方案,它避免了原始响应的全局根变量。
有关类似的更强大的Python 3解决方案,请参见本要点,该解决方案使用元类注册自定义构造函数。
import yaml
import os
class Loader(yaml.SafeLoader):
def __init__(self, stream):
self._root = os.path.split(stream.name)[0]
super(Loader, self).__init__(stream)
def include(self, node):
filename = os.path.join(self._root, self.construct_scalar(node))
with open(filename, 'r') as f:
return yaml.load(f, Loader)
Loader.add_constructor('!include', Loader.include)
一个例子:
foo.yaml
a: 1
b:
- 1.43
- 543.55
c: !include bar.yaml
bar.yaml
- 3.6
- [1, 2, 3]
现在可以使用以下命令加载文件:
>>> with open('foo.yaml', 'r') as f:
>>> data = yaml.load(f, Loader)
>>> data
{'a': 1, 'b': [1.43, 543.55], 'c': [3.6, [1, 2, 3]]}
include
功能代码可以简化:`def include(loader,node):“”“包含另一个YAML文件。”“” filename = loader.construct_scalar(node)data = yaml.load(open(filename))`
a: bla
bar.yaml:`!include foo.yaml b:blubb`这样结果将是:'{'a':bla,'b':blubb}
如果您使用的是Symfony的YAML版本,则可以这样,如下所示:
imports:
- { resource: sub-directory/file.yml }
- { resource: sub-directory/another-file.yml }
据我所知,YAML不直接支持Includes,您必须自己提供一种机制,但是,这通常很容易做到。
我已经在我的python应用程序中将YAML用作配置语言,并且在这种情况下经常定义如下约定:
>>> main.yml <<<
includes: [ wibble.yml, wobble.yml]
然后在我的(python)代码中执行以下操作:
import yaml
cfg = yaml.load(open("main.yml"))
for inc in cfg.get("includes", []):
cfg.update(yaml.load(open(inc)))
唯一的缺点是include中的变量将始终覆盖main中的变量,并且无法通过更改“ includes:”语句在main.yml文件中的位置来更改优先级。
略有不同的是,YAML不支持包含,因为它的设计并不像基于文件的标记那样专门。如果您是在响应AJAX请求时得到的,包含将意味着什么?
对于Python用户,您可以尝试pyyaml-include。
pip install pyyaml-include
import yaml
from yamlinclude import YamlIncludeConstructor
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.FullLoader, base_dir='/your/conf/dir')
with open('0.yaml') as f:
data = yaml.load(f, Loader=yaml.FullLoader)
print(data)
考虑我们有这样的YAML文件:
├── 0.yaml
└── include.d
├── 1.yaml
└── 2.yaml
1.yaml
的内容:name: "1"
2.yaml
的内容:name: "2"
在顶层:
如果0.yaml
是:
!include include.d/1.yaml
我们会得到:
{"name": "1"}
在映射中:
如果0.yaml
是:
file1: !include include.d/1.yaml
file2: !include include.d/2.yaml
我们会得到:
file1:
name: "1"
file2:
name: "2"
按顺序:
如果0.yaml
是:
files:
- !include include.d/1.yaml
- !include include.d/2.yaml
我们会得到:
files:
- name: "1"
- name: "2"
ℹ 注意:
文件名可以是绝对的(如
/usr/conf/1.5/Make.yml
)或相对的(如../../cfg/img.yml
)。
文件名可以包含shell样式的通配符。从通配符找到的文件中加载的数据将按顺序设置。
如果0.yaml
是:
files: !include include.d/*.yaml
我们会得到:
files:
- name: "1"
- name: "2"
ℹ 注意:
- 对于
Python>=3.5
,如果YAML标签的recursive
参数为,则该模式将匹配任何文件以及零个或多个目录和子目录。!include
true
“**”
“**”
由于递归搜索,在大型目录树中使用该模式可能会花费大量时间。
为了启用recursive
参数,我们!include
将以Mapping
或Sequence
模式写入标记:
Sequence
模式下的参数:!include [tests/data/include.d/**/*.yaml, true]
Mapping
模式下的参数:!include {pathname: tests/data/include.d/**/*.yaml, recursive: true}
扩展@Josh_Bode的答案,这是我自己的PyYAML解决方案,该解决方案的优点是是的自包含子类yaml.Loader
。它不依赖于任何模块级别的全局变量,也不依赖于修改yaml
模块的全局状态。
import yaml, os
class IncludeLoader(yaml.Loader):
"""
yaml.Loader subclass handles "!include path/to/foo.yml" directives in config
files. When constructed with a file object, the root path for includes
defaults to the directory containing the file, otherwise to the current
working directory. In either case, the root path can be overridden by the
`root` keyword argument.
When an included file F contain its own !include directive, the path is
relative to F's location.
Example:
YAML file /home/frodo/one-ring.yml:
---
Name: The One Ring
Specials:
- resize-to-wearer
Effects:
- !include path/to/invisibility.yml
YAML file /home/frodo/path/to/invisibility.yml:
---
Name: invisibility
Message: Suddenly you disappear!
Loading:
data = IncludeLoader(open('/home/frodo/one-ring.yml', 'r')).get_data()
Result:
{'Effects': [{'Message': 'Suddenly you disappear!', 'Name':
'invisibility'}], 'Name': 'The One Ring', 'Specials':
['resize-to-wearer']}
"""
def __init__(self, *args, **kwargs):
super(IncludeLoader, self).__init__(*args, **kwargs)
self.add_constructor('!include', self._include)
if 'root' in kwargs:
self.root = kwargs['root']
elif isinstance(self.stream, file):
self.root = os.path.dirname(self.stream.name)
else:
self.root = os.path.curdir
def _include(self, loader, node):
oldRoot = self.root
filename = os.path.join(self.root, loader.construct_scalar(node))
self.root = os.path.dirname(filename)
data = yaml.load(open(filename, 'r'))
self.root = oldRoot
return data
yaml.load(f, IncludeLoader)
在内部使用,_include
则可以避免替换根。另外,除非您执行此操作,否则解决方案的工作深度不会超过一层,因为所包含的数据使用常规yaml.Loader
类。
root
的kwargs
设置后,self.root
把它处理字符串。我将if-else块移到了super
调用上方。也许其他人可以确认我的发现或向我展示如何使用带有字符串和root
参数的类。
我举一些例子供您参考。
import yaml
main_yaml = """
Package:
- !include _shape_yaml
- !include _path_yaml
"""
_shape_yaml = """
# Define
Rectangle: &id_Rectangle
name: Rectangle
width: &Rectangle_width 20
height: &Rectangle_height 10
area: !product [*Rectangle_width, *Rectangle_height]
Circle: &id_Circle
name: Circle
radius: &Circle_radius 5
area: !product [*Circle_radius, *Circle_radius, pi]
# Setting
Shape:
property: *id_Rectangle
color: red
"""
_path_yaml = """
# Define
Root: &BASE /path/src/
Paths:
a: &id_path_a !join [*BASE, a]
b: &id_path_b !join [*BASE, b]
# Setting
Path:
input_file: *id_path_a
"""
# define custom tag handler
def yaml_import(loader, node):
other_yaml_file = loader.construct_scalar(node)
return yaml.load(eval(other_yaml_file), Loader=yaml.SafeLoader)
def yaml_product(loader, node):
import math
list_data = loader.construct_sequence(node)
result = 1
pi = math.pi
for val in list_data:
result *= eval(val) if isinstance(val, str) else val
return result
def yaml_join(loader, node):
seq = loader.construct_sequence(node)
return ''.join([str(i) for i in seq])
def yaml_ref(loader, node):
ref = loader.construct_sequence(node)
return ref[0]
def yaml_dict_ref(loader: yaml.loader.SafeLoader, node):
dict_data, key, const_value = loader.construct_sequence(node)
return dict_data[key] + str(const_value)
def main():
# register the tag handler
yaml.SafeLoader.add_constructor(tag='!include', constructor=yaml_import)
yaml.SafeLoader.add_constructor(tag='!product', constructor=yaml_product)
yaml.SafeLoader.add_constructor(tag='!join', constructor=yaml_join)
yaml.SafeLoader.add_constructor(tag='!ref', constructor=yaml_ref)
yaml.SafeLoader.add_constructor(tag='!dict_ref', constructor=yaml_dict_ref)
config = yaml.load(main_yaml, Loader=yaml.SafeLoader)
pk_shape, pk_path = config['Package']
pk_shape, pk_path = pk_shape['Shape'], pk_path['Path']
print(f"shape name: {pk_shape['property']['name']}")
print(f"shape area: {pk_shape['property']['area']}")
print(f"shape color: {pk_shape['color']}")
print(f"input file: {pk_path['input_file']}")
if __name__ == '__main__':
main()
输出
shape name: Rectangle
shape area: 200
shape color: red
input file: /path/src/a
可以像这样组合
# xxx.yaml
CREATE_FONT_PICTURE:
PROJECTS:
SUNG: &id_SUNG
name: SUNG
work_dir: SUNG
output_dir: temp
font_pixel: 24
DEFINE: &id_define !ref [*id_SUNG] # you can use config['CREATE_FONT_PICTURE']['DEFINE'][name, work_dir, ... font_pixel]
AUTO_INIT:
basename_suffix: !dict_ref [*id_define, name, !product [5, 3, 2]] # SUNG30
# ↓ This is not correct.
# basename_suffix: !dict_ref [*id_define, name, !product [5, 3, 2]] # It will build by Deep-level. id_define is Deep-level: 2. So you must put it after 2. otherwise, it can't refer to the correct value.
不幸的是,YAML没有在其标准中提供此功能。
但是,如果您使用的是Ruby,则可以通过扩展ruby YAML库来提供所需的功能:https : //github.com/entwanderer/yaml_extend
我认为@ maxy-B使用的解决方案看起来很棒。但是,对于嵌套包容而言,它对我而言并不成功。例如,如果config_1.yaml包含config_2.yaml,其中包括config_3.yaml,则加载程序存在问题。但是,如果您只是在加载时将新的加载程序类指向自身,它就可以工作!具体来说,如果我们用稍作修改的版本替换旧的_include函数:
def _include(self, loader, node):
oldRoot = self.root
filename = os.path.join(self.root, loader.construct_scalar(node))
self.root = os.path.dirname(filename)
data = yaml.load(open(filename, 'r'), loader = IncludeLoader)
self.root = oldRoot
return data
经过反思,我同意其他意见,即嵌套加载通常不适用于yaml,因为输入流可能不是文件,但它非常有用!
也许这可以激发您的灵感,请尝试与jbb约定保持一致:
https://docs.openstack.org/infra/jenkins-job-builder/definition.html#inclusion-tags
- job:
name: test-job-include-raw-1
builders:
- shell:
!include-raw: include-raw001-hello-world.sh
标准YAML 1.2本身不包含此功能。尽管如此,许多实现还是为此提供了一些扩展。
我提出了一种使用Java和snakeyaml:1.24
(用于解析/发出YAML文件的Java库)实现此目标的方法,该方法允许创建自定义YAML标签来实现以下目标(您将看到我正在使用它来加载在多个YAML文件中定义的测试套件)并且我将其作为目标test:
节点的包含列表使用):
# ... yaml prev stuff
tests: !include
- '1.hello-test-suite.yaml'
- '3.foo-test-suite.yaml'
- '2.bar-test-suite.yaml'
# ... more yaml document
这是允许处理!include
标记的一类Java 。文件从类路径(Maven资源目录)加载:
/**
* Custom YAML loader. It adds support to the custom !include tag which allows splitting a YAML file across several
* files for a better organization of YAML tests.
*/
@Slf4j // <-- This is a Lombok annotation to auto-generate logger
public class MyYamlLoader {
private static final Constructor CUSTOM_CONSTRUCTOR = new MyYamlConstructor();
private MyYamlLoader() {
}
/**
* Parse the only YAML document in a stream and produce the Java Map. It provides support for the custom !include
* YAML tag to split YAML contents across several files.
*/
public static Map<String, Object> load(InputStream inputStream) {
return new Yaml(CUSTOM_CONSTRUCTOR)
.load(inputStream);
}
/**
* Custom SnakeYAML constructor that registers custom tags.
*/
private static class MyYamlConstructor extends Constructor {
private static final String TAG_INCLUDE = "!include";
MyYamlConstructor() {
// Register custom tags
yamlConstructors.put(new Tag(TAG_INCLUDE), new IncludeConstruct());
}
/**
* The actual include tag construct.
*/
private static class IncludeConstruct implements Construct {
@Override
public Object construct(Node node) {
List<Node> inclusions = castToSequenceNode(node);
return parseInclusions(inclusions);
}
@Override
public void construct2ndStep(Node node, Object object) {
// do nothing
}
private List<Node> castToSequenceNode(Node node) {
try {
return ((SequenceNode) node).getValue();
} catch (ClassCastException e) {
throw new IllegalArgumentException(String.format("The !import value must be a sequence node, but " +
"'%s' found.", node));
}
}
private Object parseInclusions(List<Node> inclusions) {
List<InputStream> inputStreams = inputStreams(inclusions);
try (final SequenceInputStream sequencedInputStream =
new SequenceInputStream(Collections.enumeration(inputStreams))) {
return new Yaml(CUSTOM_CONSTRUCTOR)
.load(sequencedInputStream);
} catch (IOException e) {
log.error("Error closing the stream.", e);
return null;
}
}
private List<InputStream> inputStreams(List<Node> scalarNodes) {
return scalarNodes.stream()
.map(this::inputStream)
.collect(toList());
}
private InputStream inputStream(Node scalarNode) {
String filePath = castToScalarNode(scalarNode).getValue();
final InputStream is = getClass().getClassLoader().getResourceAsStream(filePath);
Assert.notNull(is, String.format("Resource file %s not found.", filePath));
return is;
}
private ScalarNode castToScalarNode(Node scalarNode) {
try {
return ((ScalarNode) scalarNode);
} catch (ClassCastException e) {
throw new IllegalArgumentException(String.format("The value must be a scalar node, but '%s' found" +
".", scalarNode));
}
}
}
}
}
使用Symfony,其对yaml的处理将间接允许您嵌套yaml文件。诀窍是利用该parameters
选项。例如:
普通版
parameters:
yaml_to_repeat:
option: "value"
foo:
- "bar"
- "baz"
配置文件
imports:
- { resource: common.yml }
whatever:
thing: "%yaml_to_repeat%"
other_thing: "%yaml_to_repeat%"
结果将与以下内容相同:
whatever:
thing:
option: "value"
foo:
- "bar"
- "baz"
other_thing:
option: "value"
foo:
- "bar"
- "baz"
提出问题时可能不支持该文件,但是您可以将其他YAML文件导入到一个文件中:
imports: [/your_location_to_yaml_file/Util.area.yaml]
尽管我没有任何在线参考,但这对我有用。
imports
。