MissingTokenException

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

MissingTokenException

Wilbur Lang
Hi,

I'm working on ANTLR to create a parser but failed with
MissingTokenException. Following is the grammar:


grammar test1;

moduleId
     :    TYPEREFERENCE
     ;

WS  :   ( ' '| '\t'| '\r'| '\n') {$channel=HIDDEN;}
     ;

fragment UPPER :    ('A'..'Z') ;
fragment LOWER :    ('a'..'z') ;
fragment DIGIT :    ('0'..'9') ;
HYPHEN :    '-' ;

CAPITALREFERENCE
     :    UPPER (UPPER|DIGIT)* (HYPHEN (UPPER|DIGIT)+)*
     ;

TYPEREFERENCE
     :    UPPER (UPPER|LOWER|DIGIT)* (HYPHEN (UPPER|LOWER|DIGIT)+)*
     ;

When I input sentence 'T3', ANTLR failed with 'MissingTokenException'.
When I comment 'CAPITALREFERENCE', ANTLR worked. How can I correct the
grammar?

Please note, I need 'CAPITALREFERENCE' to distinguish such as 'Tabc-01'
and 'TABC-01'.

Thanks.

Wilbur Lang

--------------------------------------------------------------
Ovi Mail: Making email access easy
http://mail.ovi.com


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Reply | Threaded
Open this post in threaded view
|

Re: MissingTokenException

Kevin J. Cummings
On 02/16/2011 10:38 AM, Wilbur Lang wrote:

> Hi,
>
> I'm working on ANTLR to create a parser but failed with
> MissingTokenException. Following is the grammar:
>
>
> grammar test1;
>
> moduleId
>      :    TYPEREFERENCE
>      ;
>
> WS  :   ( ' '| '\t'| '\r'| '\n') {$channel=HIDDEN;}
>      ;
>
> fragment UPPER :    ('A'..'Z') ;
> fragment LOWER :    ('a'..'z') ;
> fragment DIGIT :    ('0'..'9') ;
> HYPHEN :    '-' ;
>
> CAPITALREFERENCE
>      :    UPPER (UPPER|DIGIT)* (HYPHEN (UPPER|DIGIT)+)*
>      ;
>
> TYPEREFERENCE
>      :    UPPER (UPPER|LOWER|DIGIT)* (HYPHEN (UPPER|LOWER|DIGIT)+)*
>      ;
>
> When I input sentence 'T3', ANTLR failed with 'MissingTokenException'.
> When I comment 'CAPITALREFERENCE', ANTLR worked. How can I correct the
> grammar?

I'd do the following:

fragment TYPEREFERENCE : ;

CAPITALREFERENCE
    : UPPER (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })*
     (HYPHEN (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })+ )*
    ;

That way you'll only change the token type to TYPEREFERENCE if 1 or more
LOWER characters are found....

> Please note, I need 'CAPITALREFERENCE' to distinguish such as 'Tabc-01'
> and 'TABC-01'.
>
> Thanks.
>
> Wilbur Lang

--
Kevin J. Cummings
[hidden email]
[hidden email]
[hidden email]
Registered Linux User #1232 (http://counter.li.org)

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Reply | Threaded
Open this post in threaded view
|

Re: MissingTokenException

Wilbur Lang
On 2011/2/16 23:54, Kevin J. Cummings wrote:

> On 02/16/2011 10:38 AM, Wilbur Lang wrote:
>> moduleId
>>       :    TYPEREFERENCE
>>       ;
>>
>> I'd do the following:
>>
>> fragment TYPEREFERENCE : ;
>>
>> CAPITALREFERENCE
>>      : UPPER (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })*
>>       (HYPHEN (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })+ )*
>>      ;
>>
>> That way you'll only change the token type to TYPEREFERENCE if 1 or more
>> LOWER characters are found....
Thanks Kevin.

I rechecked the grammar, it not a context-free one, and it's impossible
to disambiguate 'T3' is a CAPTIALREFERENCE or TYPEREFERENCE. Maybe I
should check the 'semantic predicated'.

Wilbur Lang

--------------------------------------------------------------
Ovi Mail: Making email access easy
http://mail.ovi.com


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Reply | Threaded
Open this post in threaded view
|

Re: MissingTokenException

Jim Idle
Kevin's suggestion is the best way to do this as it involves no
predicates.

Jim

> -----Original Message-----
> From: [hidden email] [mailto:antlr-interest-
> [hidden email]] On Behalf Of Wilbur Lang
> Sent: Wednesday, February 16, 2011 9:14 AM
> To: [hidden email]
> Subject: Re: [antlr-interest] MissingTokenException
>
> On 2011/2/16 23:54, Kevin J. Cummings wrote:
> > On 02/16/2011 10:38 AM, Wilbur Lang wrote:
> >> moduleId
> >>       :    TYPEREFERENCE
> >>       ;
> >>
> >> I'd do the following:
> >>
> >> fragment TYPEREFERENCE : ;
> >>
> >> CAPITALREFERENCE
> >>      : UPPER (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })*
> >>       (HYPHEN (UPPER | DIGIT | LOWER { $type=TYPEREFERENCE; })+ )*
> >>      ;
> >>
> >> That way you'll only change the token type to TYPEREFERENCE if 1 or
> >> more LOWER characters are found....
> Thanks Kevin.
>
> I rechecked the grammar, it not a context-free one, and it's impossible
> to disambiguate 'T3' is a CAPTIALREFERENCE or TYPEREFERENCE. Maybe I
> should check the 'semantic predicated'.
>
> Wilbur Lang
>
> --------------------------------------------------------------
> Ovi Mail: Making email access easy
> http://mail.ovi.com
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address