From:	CRDGW2::CRDGW2::MRGATE::"SMTP::CRVAX.SRI.COM::RELAY-INFO-VAX" 30-AUG-1989 03:24
To:	MRGATE::"ARISIA::EVERHART"
Subj:	Re: Flex (latest 2.1 beta) on VMS

Received: From KL.SRI.COM by CRVAX.SRI.COM with TCP; Tue, 29 AUG 89 23:48:25 PDT
Received: from ucbvax.Berkeley.EDU by KL.SRI.COM with TCP; Tue, 29 Aug 89 22:59:13 PDT
Received: by ucbvax.Berkeley.EDU (5.61/1.37)
	id AA03814; Tue, 29 Aug 89 22:49:53 -0700
Received: from USENET by ucbvax.Berkeley.EDU with netnews
	for info-vax@kl.sri.com (info-vax@kl.sri.com)
	(contact usenet@ucbvax.Berkeley.EDU if you have questions)
Date: 29 Aug 89 22:21:30 GMT
From: att!occrsh!jct!@ucbvax.Berkeley.EDU  (jct)
Organization: Kerr-McGee, Okla. City
Subject: Re: Flex (latest 2.1 beta) on VMS
Message-Id: <291@jct.UUCP>
References: <13603@bloom-beacon.MIT.EDU>, <1664@naucse.UUCP>
Sender: info-vax-request@kl.sri.com
To: info-vax@kl.sri.com

In article <1664@naucse.UUCP>, jdc@naucse.UUCP (John Campbell) writes:
> From article <13603@bloom-beacon.MIT.EDU>, by scs@adam.pika.mit.edu (Steve Summit):
> : In article <1629@naucse.UUCP> jdc@naucse.UUCP (John Campbell) writes:
> :>The new flex reads a large chunck at a
> :>time.  With VMS STREAM-LF files this works just fine--but with "normal"
>
> Stuff deleted
>
> : This sounds like a bug in flex.  If I understand the complaint
> 
> I'm the original poster of the first article.  To date I have not proven
> that there is a bug in flex.  I still believe this is true, but I haven't
> had the time to make the obvious tests in other environments.  I still
> believe that yyunput() is in error in some way, but I would like to have
> some follow up information before I bug Vern Paxson with my worries.
> 
> Stuff deleted
>

I noticed the same problems. I traced it to a macro substitution occuring
at the beginning of a read() buffer. The substitustion uses yyunput() to do
the actual change. Since on VMS this is the beginning of a line it shows up 
more easily than on UNIX. I say its a flat out bug in yyunput(), I changed 
to the following (which is less CPU efficient but works):

#ifdef __STDC__
void yyunput (int c, register char *yy_bp)
#else
void yyunput (c, yy_bp)
  int           c;
  register char *yy_bp;
#endif
{
  register char *yy_cp = yy_c_buf_p;

  *yy_cp = yy_hold_char;		/* undo effects of setting up yytext */
  if (yy_cp < yy_ch_buf + 2)
    { 					/* need to shift up to make room */
      register int number_to_move = yy_n_chars + 2;	/* +2 for EOB chars */
      register char *source = &yy_ch_buf [number_to_move];
      register char *dest = source + 1;

      while (source > yy_ch_buf)
	*--dest = *--source;
      yy_cp++;
      yy_bp++;
      yy_n_chars++;
      if (yy_cp < yy_ch_buf + 2)
	YY_FATAL_ERROR ("flex scanner push-back overflow");
    }
  if (yy_cp > yy_bp && yy_cp [-1] == '\n')
    yy_cp [-2] = '\n';
  *--yy_cp = c;
  YY_DO_BEFORE_ACTION;			/* set up yytext again */
}

That is instead of moving the whole buffer up to the end, just move up
to make room for 1 more character. I've had no problems since.

John Tompkins
occrsh!jct!jct

