c - Why does this bison code produce unexpected output? -

flex code:

  1 %option noyywrap nodefault yylineno case-insensitive   2 %{   3 #include "stdio.h"   4 #include "tp.tab.h"   5 %}   6    7 %%   8 "{"             {return '{';}   9 "}"             {return '}';}  10 ";"             {return ';';}  11 "create"        {return create;}  12 "cmd"           {return cmd;}  13 "int"           {yylval.intval = 20;return int;}  14 [a-za-z]+       {yylval.strval = yytext;printf("id:%s\n" , yylval.strval);return id;}  15 [ \t\n]  16 <<eof>>         {return 0;}  17 .               {printf("mistery char\n");}  18  

bison code:

  1 %{   2 #include "stdlib.h"   3 #include "stdio.h"   4 #include "stdarg.h"   5 void yyerror(char *s, ...);   6 #define yydebug 1   7 int yydebug = 1;   8 %}   9   10 %union{  11     char *strval;  12     int intval;  13 }  14   15 %token <strval> id  16 %token <intval> int  17 %token create  18 %token cmd  19   20 %type <strval> col_definition  21 %type <intval> create_type  22 %start stmt_list  23   24 %%  25 stmt_list:stmt ';'  26 | stmt_list stmt ';'  27 ;  28   29 stmt:create_cmd_stmt         {/*printf("create cmd\n");*/}  30 ;  31   32 create_cmd_stmt:create cmd id'{'create_col_list'}'    {printf("%s\n" , $3);}  33 ;  34 create_col_list:col_definition  35 | create_col_list col_definition  36 ;  37   38 col_definition:create_type id ';' {printf("%d , %s\n" , $1, $2);}  39 ;  40   41 create_type:int {$$ = $1;}  42 ;  43   44 %%  45 extern file *yyin;  46   47 void  48 yyerror(char *s, ...)  49 {  50     extern yylineno;  51     va_list ap;  52     va_start(ap, s);  53     fprintf(stderr, "%d: error: ", yylineno);  54     vfprintf(stderr, s, ap);  55     fprintf(stderr, "\n");  56 }  57   58 int main(int argc , char *argv[])  59 {  60     yyin = fopen(argv[1] , "r");  61     if(!yyin){  62         printf("open file %s failed\n" ,argv[1]);  63         return -1;  64     }  65   66     if(!yyparse()){  67         printf("parse work!\n");  68     }else{  69         printf("parse failed!\n");  70     }  71   72     fclose(yyin);  73     return 0;  74 }  75 

test input file:

create cmd keeplive {     int a;     int b; }; 

test output:

root@vm-ubuntu203001:~/test/tpp# ./a.out t1.tp  id:keeplive id:a 20 , a; id:b 20 , b; keeplive {     int a;     int b; } parse work! 

i have 2 questions:

1) why action @ line 38 print token ';'? instance, "20 , a;" , "20 , b;"

2) why action @ line 32 print "keeplive { int a; int b; }" instead of "keeplive"?

short answer:

yylval.strval = yytext; 

you can't use yytext that. string points private lexer , change flex action finishes. need like:

yylval.strval = strdup(yytext); 

and need make sure free memory afterwards.

---

longer answer:

yytext pointer buffer containing input. in order make yytext work though nul-terminated string, flex framework overwrites character following token nul before action, , replaces original character when action terminates. strdup work fine inside action, outside action (in bison code), have pointer part of buffer starting token. , gets worse later, since flex read next part of source same buffer, , pointer random garbage. there several possible scenarios, depending on flex options, none of them pretty.

so golden rule: yytext valid until end of action. if want keep it, copy it, , make sure free storage copy when no longer need it.

in lexers i've written, id token finds identifier in symbol table (or puts there) , returns pointer symbol table, simplifies memory management. still have same memory management issue with, example, character string literals.