Spirit解析器分段错误
Spirit parser segfaults
当我运行这个时,我得到一个段错误。它看起来像调试打印,但当我调试它时,我只是得到一个无限循环的回溯。如果有人能帮我指出正确的方向,我会很感激的。我也很感激,如果可能的话,任何清理这个的提示/技巧语法。
谢谢!
//code here:
/***
*I-EBNF parser
*
*This defines a grammar for BNF.
*/
//Speeds up compilation times.
//This is a relatively small grammar, this is useful.
#define BOOST_SPIRIT_NO_PREDEFINED_TERMINALS
#define BOOST_SPIRIT_QI_DEBUG
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/fusion/adapted.hpp>
#include <boost/fusion/support.hpp>
#include <vector>
#include <string>
#include <iostream>
namespace Parser
{
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
enum class RHSType
{
Terminal, Identifier
};
struct RHS
{
RHSType type;
std::string value;
};
struct Rule
{
std::string identifier; //lhs
std::vector<RHS> rhs;
};
}
//expose our structs to fusion:
BOOST_FUSION_ADAPT_STRUCT(
Parser::RHS,
(Parser::RHSType, type)
(std::string, value)
)
BOOST_FUSION_ADAPT_STRUCT(
Parser::Rule,
(std::string, identifier)
(std::vector<Parser::RHS>, rhs)
)
namespace Parser
{
typedef std::vector<Rule> RuleList;
//our grammar definition
template <typename Iterator>
struct Grammar: qi::grammar<Iterator, std::list<Rule>, ascii::space_type>
{
Grammar(): Grammar::base_type(rules)
{
qi::char_type char_;
letter = char_("a-zA-Z");
digit = char_('0', '9');
symbol = char_('[') | ']' | '[' | ']' | '(' | ')' | '<' | '>'
| ''' | '"' | '=' | '|' | '.' | ',' | ';';
character = letter | digit | symbol | '_';
identifier = letter >> *(letter | digit | '_');
terminal = (char_(''') >> character >> *character >>
char_(''')) | (char_('"') >> character >> *character >> char_('"'));
lhs = identifier;
rhs = terminal | identifier | char_('[') >> rhs >> char_(']')
| char_('{') >> rhs >> char_('}') | char_('(') >> rhs >> char_(')') |
rhs >> char_('|') >> rhs | rhs >> char_(',') >> rhs;
rule = identifier >> char_('=') >> rhs;
rules = rule >> *rule;
}
private:
qi::rule<Iterator, char(), ascii::space_type> letter, digit,
symbol, character;
qi::rule<Iterator, std::string(), ascii::space_type> identifier,
lhs, terminal;
qi::rule<Iterator, RHS, ascii::space_type> rhs;
qi::rule<Iterator, Rule, ascii::space_type> rule;
qi::rule<Iterator, std::list<Rule>, ascii::space_type> rules;
};
}
int main()
{
Parser::Grammar<std::string::const_iterator> parser;
boost::spirit::ascii::space_type space;
std::string input;
std::vector<std::string> output;
bool result;
while (std::getline(std::cin, input))
{
if (input.empty())
{
break;
}
std::string::const_iterator it, itEnd;
it = input.begin();
itEnd = input.end();
result = phrase_parse(it, itEnd, parser, space, output);
if (result && it == itEnd)
{
std::cout << "success" << std::endl;
}
}
return 0;
}
¹交叉张贴从[spirit-general]邮件列表:http://boost.2283326.n4.nabble.com/parser-segfault-tips-tricks-td4680336.html
2015年9月26日01:45 AM, Littlefield, Tyler写道:
你好:当我运行这个时,我得到了一个分段错误。它看起来像调试打印,但当我调试它时,我只是得到一个无限循环的回溯。如果有人能帮我指出正确的方向,我会很感激的。我也很感激,如果可能的话,任何清理这个的提示/技巧语法。
首先,它不能编译。
它不应该编译,因为语法不公开属性(你是说list<Rule>()
而不是list<Rule>
吗?)
但是你永远不能把它赋值给output
变量(也就是std::vector<std::string>
?!?)
同样地,你也忘了这里的括号
qi::rule<Iterator, RHS(), ascii::space_type> rhs;
qi::rule<Iterator, Rule(), ascii::space_type> rule;
qi::rule<Iterator, std::list<Rule>(), ascii::space_type> rules;
rhs规则具有无限左递归:
rhs = terminal
| identifier
| ('[' >> rhs >> ']')
| ('{' >> rhs >> '}')
| ('(' >> rhs >> ')')
| (rhs >> '|' >> rhs) // OOPS
| (rhs >> ',' >> rhs) // OOPS
;
这可能解释了崩溃的原因,因为它会导致stackoverflow。
指出直播录音(第1部分和第2部分)准确地展示了我首先清理语法并随后使其真正具有编译价值的步骤。
那里有相当多的工作:
- 清理:对中断(
[]
,{}
,()
和=
,|
,,
)使用隐式qi::lit
- 使用kleene+代替
a >> *a
(不止一次) - 首选
%
解析器来解析…列出 我不得不围绕"RHS"稍微"扭动"一下规则;在最后两个分支中存在无限递归(见
// OOPS
)。我通过引入一个"纯"表达式规则(它只解析一个RHS
结构)来修复它。我已将此类型重命名为Expression
。"列表"解析(识别由
,
或|
分隔的表达式列表)被移到原始的rhs
规则中,为了更具描述性,我将其重命名为expr_list
:expression = qi::attr(Parser::ExprType::Terminal) >> terminal | qi::attr(Parser::ExprType::Identifier) >> identifier | qi::attr(Parser::ExprType::Compound) >> qi::raw [ '[' >> expr_list >> ']' ] | qi::attr(Parser::ExprType::Compound) >> qi::raw [ '{' >> expr_list >> '}' ] | qi::attr(Parser::ExprType::Compound) >> qi::raw [ '(' >> expr_list >> ')' ] ; expr_list = expression % (char_("|,")) // TODO FIXME? ;
为了将合成属性实际转换为
RHS
(现在:Expression
)类型,我们需要为第一个适应的成员实际公开RHSType
(现在:ExprType
)值。您可以在上面的行中看到,我们为此使用了qi::attr()
。现代编译器和boost版本可以大大简化
BOOST_FUSION_ADAPT_STRUCT
调用:BOOST_FUSION_ADAPT_STRUCT(Parser::Expression, type, value) BOOST_FUSION_ADAPT_STRUCT(Parser::Rule, identifier, expr_list)
我将一些规则"升级"为lexeme ,这意味着它们不服从船长。
我猜测我可能应该将空格字符添加到
terminal
(字符串文字)中可接受的字符集中。如果这不是你想要的,只要从char_("[][]()<>'"=|.,;_ ");
中删除最后一个字符。我还将skipper更改为
blank_type
,因为它不会跳过换行字符。您可以使用相同的语法轻松地直接解析多行输入。做的,例如:rules = rule % qi::eol;
参见:关于船长,词素及其相互作用的附加信息的增强精神船长问题。
话不多说,下面是一个工作示例:
Live On Coliru
#define BOOST_SPIRIT_DEBUG
#include <boost/fusion/adapted.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/qi.hpp>
#include <iostream>
#include <string>
#include <vector>
namespace Parser
{
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
enum class ExprType { Terminal, Identifier, Compound };
static inline std::ostream& operator<<(std::ostream& os, ExprType type) {
switch (type) {
case ExprType::Terminal: return os << "Terminal";
case ExprType::Identifier: return os << "Identifier";
case ExprType::Compound: return os << "Compound";
}
return os << "(unknown)";
}
struct Expression { // TODO make recursive (see `boost::make_recursive_variant`)
ExprType type;
std::string value;
};
using ExprList = std::vector<Expression>;
struct Rule {
std::string identifier; // lhs
ExprList expr_list;
};
}
//expose our structs to fusion:
BOOST_FUSION_ADAPT_STRUCT(Parser::Expression, type, value)
BOOST_FUSION_ADAPT_STRUCT(Parser::Rule, identifier, expr_list)
namespace Parser
{
typedef std::list<Rule> RuleList;
//our grammar definition
template <typename Iterator>
struct Grammar: qi::grammar<Iterator, RuleList(), ascii::blank_type>
{
Grammar(): Grammar::base_type(rules)
{
qi::char_type char_;
symbol = char_("[][]()<>'"=|.,;_ ");
character = qi::alpha | qi::digit | symbol;
identifier = qi::alpha >> *(qi::alnum | char_('_'));
// TODO capture strings including interpunction(?)
terminal = (''' >> +(character - ''') >> ''')
| ('"' >> +(character - '"') >> '"');
expression = qi::attr(Parser::ExprType::Terminal) >> terminal
| qi::attr(Parser::ExprType::Identifier) >> identifier
| qi::attr(Parser::ExprType::Compound) >> qi::raw [ '[' >> expr_list >> ']' ]
| qi::attr(Parser::ExprType::Compound) >> qi::raw [ '{' >> expr_list >> '}' ]
| qi::attr(Parser::ExprType::Compound) >> qi::raw [ '(' >> expr_list >> ')' ]
;
expr_list = expression % (char_("|,")) // TODO FIXME?
;
// above accepts mixed separators:
// a, b, c | d, e
//
// original accepted:
//
// a, b, [ c | d ], e
// a| b| [ c , d ]| e
// a| b| [ c | d ]| e
// a, b, [ c , d ], e
rule = identifier >> '=' >> expr_list;
//rules = rule % qi::eol; // alternatively, parse multi-line input in one go
rules = +rule;
BOOST_SPIRIT_DEBUG_NODES((rules)(rule)(expr_list)(expression)(identifier)(terminal))
}
private:
qi::rule<Iterator, Expression(), ascii::blank_type> expression;
qi::rule<Iterator, ExprList(), ascii::blank_type> expr_list;
qi::rule<Iterator, Rule(), ascii::blank_type> rule;
qi::rule<Iterator, RuleList(), ascii::blank_type> rules;
// lexemes:
qi::rule<Iterator, std::string()> terminal, identifier;
qi::rule<Iterator, char()> symbol, character;
};
}
int main() {
using It = std::string::const_iterator;
Parser::Grammar<It> parser;
boost::spirit::ascii::blank_type blank;
std::string input;
while (std::getline(std::cin, input))
{
if (input.empty()) {
break;
}
It it = input.begin(), itEnd = input.end();
Parser::RuleList output;
bool result = phrase_parse(it, itEnd, parser, blank, output);
if (result) {
std::cout << "successn";
for (auto& rule : output) {
std::cout << "ntarget: " << rule.identifier << "n";
for (auto& rhs : rule.expr_list) {
std::cout << "rhs: " << boost::fusion::as_vector(rhs) << "n";
}
}
} else {
std::cout << "parse failedn";
}
if (it != itEnd)
std::cout << "remaining unparsed: '" << std::string(it, itEnd) << "n";
}
}
打印输出:
success
target: assigned1
rhs: (Identifier some_var)
rhs: (Terminal a 'string' value)
rhs: (Compound [ 'okay', "this", is_another, identifier, { ( "nested" ) } ])
rhs: (Terminal done)
success
target: assigned2
rhs: (Compound { a })
并且启用调试(#define BOOST_SPIRIT_DEBUG
):
<rules>
<try>assigned1 = some_var</try>
<rule>
<try>assigned1 = some_var</try>
<identifier>
<try>assigned1 = some_var</try>
<success> = some_var | "a 'st</success>
<attributes>[[a, s, s, i, g, n, e, d, 1]]</attributes>
</identifier>
<expr_list>
<try> some_var | "a 'stri</try>
<expression>
<try> some_var | "a 'stri</try>
<terminal>
<try>some_var | "a 'strin</try>
<fail/>
</terminal>
<identifier>
<try>some_var | "a 'strin</try>
<success> | "a 'string' value</success>
<attributes>[[s, o, m, e, _, v, a, r]]</attributes>
</identifier>
<success> | "a 'string' value</success>
<attributes>[[Identifier, [s, o, m, e, _, v, a, r]]]</attributes>
</expression>
<expression>
<try> "a 'string' value" </try>
<terminal>
<try>"a 'string' value" |</try>
<success> | [ 'okay', "this",</success>
<attributes>[[a, , ', s, t, r, i, n, g, ', , v, a, l, u, e]]</attributes>
</terminal>
<success> | [ 'okay', "this",</success>
<attributes>[[Terminal, [a, , ', s, t, r, i, n, g, ', , v, a, l, u, e]]]</attributes>
</expression>
<expression>
<try> [ 'okay', "this", i</try>
<terminal>
<try>[ 'okay', "this", is</try>
<fail/>
</terminal>
<identifier>
<try>[ 'okay', "this", is</try>
<fail/>
</identifier>
<expr_list>
<try> 'okay', "this", is_</try>
<expression>
<try> 'okay', "this", is_</try>
<terminal>
<try>'okay', "this", is_a</try>
<success>, "this", is_another</success>
<attributes>[[o, k, a, y]]</attributes>
</terminal>
<success>, "this", is_another</success>
<attributes>[[Terminal, [o, k, a, y]]]</attributes>
</expression>
<expression>
<try> "this", is_another,</try>
<terminal>
<try>"this", is_another, </try>
<success>, is_another, identi</success>
<attributes>[[t, h, i, s]]</attributes>
</terminal>
<success>, is_another, identi</success>
<attributes>[[Terminal, [t, h, i, s]]]</attributes>
</expression>
<expression>
<try> is_another, identif</try>
<terminal>
<try>is_another, identifi</try>
<fail/>
</terminal>
<identifier>
<try>is_another, identifi</try>
<success>, identifier, { ( "n</success>
<attributes>[[i, s, _, a, n, o, t, h, e, r]]</attributes>
</identifier>
<success>, identifier, { ( "n</success>
<attributes>[[Identifier, [i, s, _, a, n, o, t, h, e, r]]]</attributes>
</expression>
<expression>
<try> identifier, { ( "ne</try>
<terminal>
<try>identifier, { ( "nes</try>
<fail/>
</terminal>
<identifier>
<try>identifier, { ( "nes</try>
<success>, { ( "nested" ) } ]</success>
<attributes>[[i, d, e, n, t, i, f, i, e, r]]</attributes>
</identifier>
<success>, { ( "nested" ) } ]</success>
<attributes>[[Identifier, [i, d, e, n, t, i, f, i, e, r]]]</attributes>
</expression>
<expression>
<try> { ( "nested" ) } ] </try>
<terminal>
<try>{ ( "nested" ) } ] |</try>
<fail/>
</terminal>
<identifier>
<try>{ ( "nested" ) } ] |</try>
<fail/>
</identifier>
<expr_list>
<try> ( "nested" ) } ] | </try>
<expression>
<try> ( "nested" ) } ] | </try>
<terminal>
<try>( "nested" ) } ] | "</try>
<fail/>
</terminal>
<identifier>
<try>( "nested" ) } ] | "</try>
<fail/>
</identifier>
<expr_list>
<try> "nested" ) } ] | "d</try>
<expression>
<try> "nested" ) } ] | "d</try>
<terminal>
<try>"nested" ) } ] | "do</try>
<success> ) } ] | "done"</success>
<attributes>[[n, e, s, t, e, d]]</attributes>
</terminal>
<success> ) } ] | "done"</success>
<attributes>[[Terminal, [n, e, s, t, e, d]]]</attributes>
</expression>
<success> ) } ] | "done"</success>
<attributes>[[[Terminal, [n, e, s, t, e, d]]]]</attributes>
</expr_list>
<success> } ] | "done"</success>
<attributes>[[Compound, [(, , ", n, e, s, t, e, d, ", , )]]]</attributes>
</expression>
<success> } ] | "done"</success>
<attributes>[[[Compound, [(, , ", n, e, s, t, e, d, ", , )]]]]</attributes>
</expr_list>
<success> ] | "done"</success>
<attributes>[[Compound, [{, , (, , ", n, e, s, t, e, d, ", , ), , }]]]</attributes>
</expression>
<success> ] | "done"</success>
<attributes>[[[Terminal, [o, k, a, y]], [Terminal, [t, h, i, s]], [Identifier, [i, s, _, a, n, o, t, h, e, r]], [Identifier, [i, d, e, n, t, i, f, i, e, r]], [Compound, [{, , (, , ", n, e, s, t, e, d, ", , ), , }]]]]</attributes>
</expr_list>
<success> | "done"</success>
<attributes>[[Compound, [[, , ', o, k, a, y, ', ,, , ", t, h, i, s, ", ,, , i, s, _, a, n, o, t, h, e, r, ,, , i, d, e, n, t, i, f, i, e, r, ,, , {, , (, , ", n, e, s, t, e, d, ", , ), , }, , ]]]]</attributes>
</expression>
<expression>
<try> "done"</try>
<terminal>
<try>"done"</try>
<success></success>
<attributes>[[d, o, n, e]]</attributes>
</terminal>
<success></success>
<attributes>[[Terminal, [d, o, n, e]]]</attributes>
</expression>
<success></success>
<attributes>[[[Identifier, [s, o, m, e, _, v, a, r]], [Terminal, [a, , ', s, t, r, i, n, g, ', , v, a, l, u, e]], [Compound, [[, , ', o, k, a, y, ', ,, , ", t, h, i, s, ", ,, , i, s, _, a, n, o, t, h, e, r, ,, , i, d, e, n, t, i, f, i, e, r, ,, , {, , (, , ", n, e, s, t, e, d, ", , ), , }, , ]]], [Terminal, [d, o, n, e]]]]</attributes>
</expr_list>
<success></success>
<attributes>[[[a, s, s, i, g, n, e, d, 1], [[Identifier, [s, o, m, e, _, v, a, r]], [Terminal, [a, , ', s, t, r, i, n, g, ', , v, a, l, u, e]], [Compound, [[, , ', o, k, a, y, ', ,, , ", t, h, i, s, ", ,, , i, s, _, a, n, o, t, h, e, r, ,, , i, d, e, n, t, i, f, i, e, r, ,, , {, , (, , ", n, e, s, t, e, d, ", , ), , }, , ]]], [Terminal, [d, o, n, e]]]]]</attributes>
</rule>
<rule>
<try></try>
<identifier>
<try></try>
<fail/>
</identifier>
<fail/>
</rule>
<success></success>
<attributes>[[[[a, s, s, i, g, n, e, d, 1], [[Identifier, [s, o, m, e, _, v, a, r]], [Terminal, [a, , ', s, t, r, i, n, g, ', , v, a, l, u, e]], [Compound, [[, , ', o, k, a, y, ', ,, , ", t, h, i, s, ", ,, , i, s, _, a, n, o, t, h, e, r, ,, , i, d, e, n, t, i, f, i, e, r, ,, , {, , (, , ", n, e, s, t, e, d, ", , ), , }, , ]]], [Terminal, [d, o, n, e]]]]]]</attributes>
</rules>
success
target: assigned1
rhs: (Identifier some_var)
rhs: (Terminal a 'string' value)
rhs: (Compound [ 'okay', "this", is_another, identifier, { ( "nested" ) } ])
rhs: (Terminal done)
<rules>
<try>assigned2 = { a }</try>
<rule>
<try>assigned2 = { a }</try>
<identifier>
<try>assigned2 = { a }</try>
<success> = { a }</success>
<attributes>[[a, s, s, i, g, n, e, d, 2]]</attributes>
</identifier>
<expr_list>
<try> { a }</try>
<expression>
<try> { a }</try>
<terminal>
<try>{ a }</try>
<fail/>
</terminal>
<identifier>
<try>{ a }</try>
<fail/>
</identifier>
<expr_list>
<try> a }</try>
<expression>
<try> a }</try>
<terminal>
<try>a }</try>
<fail/>
</terminal>
<identifier>
<try>a }</try>
<success> }</success>
<attributes>[[a]]</attributes>
</identifier>
<success> }</success>
<attributes>[[Identifier, [a]]]</attributes>
</expression>
<success> }</success>
<attributes>[[[Identifier, [a]]]]</attributes>
</expr_list>
<success></success>
<attributes>[[Compound, [{, , a, , }]]]</attributes>
</expression>
<success></success>
<attributes>[[[Compound, [{, , a, , }]]]]</attributes>
</expr_list>
<success></success>
<attributes>[[[a, s, s, i, g, n, e, d, 2], [[Compound, [{, , a, , }]]]]]</attributes>
</rule>
<rule>
<try></try>
<identifier>
<try></try>
<fail/>
</identifier>
<fail/>
</rule>
<success></success>
<attributes>[[[[a, s, s, i, g, n, e, d, 2], [[Compound, [{, , a, , }]]]]]]</attributes>
</rules>
success
target: assigned2
rhs: (Compound { a })
失去结束代码中还有一些待办事项。实现它们需要更多的努力,但我不确定这实际上是正确的方向,所以我将等待反馈:)
剩下的基本TODO之一是在AST中表示表达式的递归性质。现在,我只是通过为嵌套的Compound表达式插入源字符串来"逃避"。
- 在某些循环内使用vector.push_back时出现分段错误
- 为什么在运行时没有向我们提供有关分段错误的更多信息?
- 如何解决gcc编译器优化导致的centos双编译器设置中的分段错误
- 当我的阵列太大时出现分段错误
- 分段错误当我试图运行程序时出错
- 在c++中初始化矩阵时出现分段错误(核心转储)
- 尝试使用集合函数时出现分段错误
- 我无法缩小此分段错误的原因
- g++的分段错误(在NaN上使用to_string两次时)
- 我是如何在这段代码中出现分段错误的
- 创建结构的数组时遇到分段错误
- 在c++中键入向量中的所有值后,得到分段错误(核心转储)
- 在 c++ 中实现 Trie 时出现分段错误
- 为什么 fstream 在打开带有格式的文件时会导致分段错误?
- 为什么我遇到分段错误?
- 动态类的分段错误(家庭作业问题)
- 分段错误 - 读取初始化指针的数组
- 如何摆脱C ++中的分段错误错误?
- 使用 CTYPE 时出现分段错误
- 为什么代码给出分段错误?