\part[ AUTOMATA]{AUTOMATA}

\chapter[ Automata and Formal Languages]{Automata and Formal Languages}

\vspace{-4mm}
Automata and formal languages play an important role in projecting and realizing compilers.
In  the first section grammars and formal languages are defined. The different grammars and 
languages are discussed based on Chomsky hierarchy.\nevindex{Chomsky, Noam} 
In the second section we deal in detail with the finite automata and the languages accepted 
by them, while in the third section the pushdown automata and the corresponding accepted 
languages are discussed. Finally, references from a rich bibliography are given.
 

\section{Languages and grammars}\label{KZalapfogalmak}
A finite and nonempty set of symbols is called an \ki{alphabet}. \inddef{alphabet}
The elements of an alphabet are \ki{letters,} but sometimes are named also \ki{symbols.}   

With the letters of an alphabet words are composed. If $a_1,a_2, \ldots, a_n\in \Sigma, 
n\ge 0,$ then  $a_1a_2\ldots a_n$ a $\Sigma$ is a \ki{word} over the alphabet $\Sigma$ \inddef{word}
(the letters $a_i$ are not necessary distinct). The number of letters of a word, with their multiplicities,  
constitutes the \ki{length} of the word.  If $w=a_1a_2\ldots a_n,$ then the length of  $w$ is
$|w|=n.$ If $n=0,$ then the word is an \ki{empty word,} which will be denoted by $\varepsilon$ 
(sometimes  $\lambda$ in other books).
The set of words over the alphabet  $\Sigma$ will be denoted by $\Sigma^*$:
\[
  \Sigma^*=\big\{a_1a_2\ldots a_n\;|\;  a_1, a_2, \ldots, 
   a_n\in \Sigma, \, n\ge 0\big\}\koz.
\]
For the set of nonempty words over $\Sigma$ the notation  
$\Sigma^+= \Sigma^*\setminus \{\varepsilon\}$ will be used. The set of words of length $n$ over 
$\Sigma$ will be denoted by  $\Sigma^n$, and   $\Sigma^0=\{\varepsilon\}.$ Then
\[ 
\Sigma^*= \Sigma^0 \cup \Sigma^1 \cup \cdots \cup \Sigma^n \cup \cdots 
\quad \textrm{ and } \quad
\Sigma^+= \Sigma^1 \cup \Sigma ^2 \cup \cdots \cup \Sigma^n \cup \cdots \koz .
\]
The words $u=a_1a_2\ldots a_m$  and $v=b_1b_2 \ldots b_n$ are equal (i.e. $u=v$), if  $m = n$ and 
$a_i=b_i, \;\; i=1,2,\ldots,n.$

We define in $\Sigma^*$ the binary operation called  \ki{concatenation.} \inddef{concatenation}
 The concatenation (or product) of the words  $u=a_1a_2\ldots a_m$ and 
$v=b_1b_2\ldots b_n$ is the word $uv=a_1a_2\ldots a_mb_1b_2\ldots b_n$. It is clear that 
 $|uv|=|u|+|v|.$ 
This operation is  associative but not commutative.  Its neutral element is $\varepsilon,$ because 
 $\varepsilon u= u\varepsilon = u$ for all $u\in \Sigma^*$. 
 $\Sigma^*$ with the concatenation is a  monoid.

We introduce the power operation.  If  $u\in \Sigma^*,$ then\inddef{word!power} $u^0=\varepsilon,$ 
 and $u^n=u^{n-1}u$ for $\;n\ge 1.$ The \ki{reversal} (or \ki{mirror image}) \inddef{reversal}\inddef{mirror image} of the word 
$u=a_1a_2\ldots a_n$ is $u^{-1}=a_na_{n-1}\ldots a_1$. The reversal of  $u$ sometimes is denoted by $u^R$ or
$\tilde{u}$. It is clear that  $\left(u^{-1} \right)^{-1}=u$ and 
$(uv)^{-1}= v^{-1}u^{-1}.$

Word   $v$ is a \ki{prefix}\inddef{prefix}  of the word $u$ if  there exists a word  $z$ such that  
$u=vz$.  If $z\not = \varepsilon$ then $v$ is a proper prefix of $u$. 
Similarly $v$ is a \ki{suffix}\inddef{suffix} of $u$ if there exists a word 
 $x$ such that $u=xv$.  The proper suffix can also be defined. Word  $v$ is a \ki{subword} of the word  
$u$ if  there are words  $p$ and $q$ such that $u=pvq.$ If $pq\not= \varepsilon$ then  $v$ is a 
\ki{proper subword.}\inddef{proper subword}\inddef{subword}

A subset $L$  of  $\Sigma^*$  is called a \ki{language} over the alphabet $\Sigma$. 
Sometimes this is called a \ki{formal language} because the words are here considered 
without any meanings. Note that $\emptyset$ is the empty language while  $\{\varepsilon\}$ is a language which contains the empty word.  
 
\subsection{Operations on languages} \index{operations!on languages}
If $L,$ $L_1,$ $L_2$ are languages over $\Sigma$ we define the following operations

\noindent
$\bullet$ \textit{union} \index{languages!union}

\qquad\qquad $L_1 \cup L_2 =\{ u\in \Sigma^* \;|\; u\in L_1 \;\textrm{ or }\; u\in L_2 \}\koz,$
 
\noindent$ \bullet$ \textit{intersection}  \index{languages!intersection}
 
\qquad\qquad $L_1 \cap L_2 =\{ u\in \Sigma^* \;|\; u\in L_1 \;\textrm{ and }\; u\in L_2 \}\koz,$
 
\noindent $\bullet$ \textit{difference}  \index{languages!difference}

\qquad\qquad $L_1 \setminus L_2 =\{ u\in \Sigma^* \;|\; u\in L_1 \;\textrm{ and }\; u\not\in L_2 \}\koz,$
 
\noindent $\bullet$ \textit{complement}\index{language!complement}
 
\qquad\qquad $ \overline{L}= \Sigma^* \setminus L \koz,$
 
\noindent $\bullet$ \textit{multiplication}  \index{languages!multiplication}

\qquad\qquad $L_1L_2= \{uv \;|\; u\in L_1, v\in L_2 \}\koz,$

\noindent $\bullet$ \textit{power}  \index{language!power}

\qquad\qquad $ L^0 =\{ \varepsilon\}, \qquad L^n= L^{n-1}L,\;$ if  $\; n\ge 1 \koz, $

\noindent $\bullet$ \textit{iteration} or 
\textit{star operation}\index{language!iteration}\index{language!star operation}

\qquad\qquad  $L^*=\displaystyle\bigcup_{i=0}^{\infty}L^i=L^0\cup L\cup L^2\cup\cdots\cup L^i\cup\cdots \koz ,$

\noindent
$\bullet$ \textit{mirror}  \index{language!mirror}
 
\qquad\qquad $ L^{-1}= \{ u^{-1}\;|\; u\in L \} \koz .$
 
\noindent We will use also the notation   $L^+$ 

\qquad\qquad  $L^+=\displaystyle\bigcup_{i=1}^{\infty}L^i=L\cup L^2\cup\cdots\cup L^i\cup\cdots \koz .$
 
   
\noindent The union, product and iteration are called  \ki{regular operations.}  \inddef{regular!operation}

\vspace{-1mm}
\subsection{Specifying languages} 
Languages can be specified in several ways. For example a language can be specified using

1) the enumeration of its words, 

\newpage
2) a property, such that all words of the language have this property but other word have not,

3) a grammar.

\vspace{-1mm}
\subsubsection{Specifying languages by listing their elements.} 
For example the following are languages

$L_1= \{\varepsilon, 0, 1 \},$ 

$L_2= \{ a, aa, aaa, ab, ba, aba\}.$

\noindent Even if we cannot enumerate the elements of an infinite set infinite languages can be 
specified by enumeration if after enumerating the first some elements we can continue the enumeration using a rule. 
 The following is such a language

$L_3= \{\varepsilon, ab, aabb, aaabbb, aaaabbbb,\ldots\}.$

\vspace{-1mm}
\subsubsection{Specifying languages by properties.}  The following sets are languages

$L_4=\{a^nb^n\; |\; n=0,1,2, \ldots \},$

$L_5 = \{ uu^{-1}\; |\; u\in \Sigma^*\},$

$L_6= \{ u\in \{ a,b\}^* \; |\; n_a(u)=n_b(u)\},$

\noindent where $n_a(u)$ denotes the number of letters $a$ in word  $u$ and  $n_b(u)$ the number of letters  $b$.

\subsubsection{Specifying  languages by grammars.}  \index{languages!specifying them}

Define the  \ki{generative grammar}\inddef{grammar}\index{grammar!generative}
or shortly the  \ki{grammar}. 

\begin{defi}A  \ki{grammar} is an ordered quadruple $G= (N,T,P,S)$, where

$\bullet$ $N$ is the alphabet of \ki{variables}\inddef{variable} 
(or \ki{nonterminal symbols}),\inddef{nonterminal symbol} 

$\bullet$ $T$ is the alphabet of \ki{terminal symbols,}\inddef{terminal symbol} where 
    $N\cap T=\emptyset$,
 
$\bullet$ $P \subseteq (N\cup T)^*N(N\cup T)^*\times (N\cup T)^*$ is a finite set, 
that is  $P$  is the finite set of \ki{productions} of the form  $(u,v)$ \inddef{production},  where $u, v \in (N\cup T)^*$ and $u$ contains at least a nonterminal symbol,

$\bullet$ $S\in N$ is the  \ki{start symbol.}\inddef{start symbol}
\end{defi}

\textit{Remarks.} Instead of the notation  $(u,v)$ sometimes $u\rightarrow v$ is used.

In the production $u \rightarrow v$ or $(u,v)$ word  $u$ is called the left-hand side of the production while  $v$ the right-hand side. If for a grammar there are more than one production with the same left-hand side, then these production  
\vspace{-1mm}
\[ u\rightarrow v_1,\; u\rightarrow v_2, \ldots, u\rightarrow v_r\quad \textrm{can be written as } \quad
  u\rightarrow v_1\;|\;v_2\;|\;\ldots \;|\; v_r \koz .\] 

We define on the set $(N\cup T)^*$ the relation called \ki{direct derivation} 
\vspace{-1mm}
\[ u \Longrightarrow v,  \qquad \textrm{if}\quad u=p_1pp_2, \quad v=p_1qp_2 
    \quad\textrm{ and }\quad (p,q)\in P \koz .\]
In fact we replace in  $u$ an appearance of the subword  $p$ by  $q$ and we get $v$. Another notations for the same relation can be  $\vdash$ or $\models$.  

If we want to emphasize the used grammar $G$, then the notation $\Longrightarrow$ 
can be replaced by $\underset{G}\Longrightarrow$. Relation   $\overset{*}{\Longrightarrow}$ 
is the reflexive and transitive 
closure of $\Longrightarrow$, while  $\overset{+}{\Longrightarrow}$ denotes its transitive closure. 
Relation $\overset{*}{\Longrightarrow}$ is called a \ki{derivation.} \inddef{derivation}

\newpage
From the definition of a reflexive and transitive relation we can deduce the following:  
$u \overset{*}{\Longrightarrow} v$, if there exist the words $w_0, w_1,\; \ldots, w_n\in (N\cup T)^*, \; n\ge 0$ and   $u=w_0,$ $\;w_0\Longrightarrow w_1,$  $w_1\;\Longrightarrow w_2,\; \ldots, 
w_{n-1}\Longrightarrow w_n,$ $\;w_n = v$. This  can be written shortly
$u=w_0 \Longrightarrow w_1 \Longrightarrow w_2 \Longrightarrow \ldots \Longrightarrow w_{n-1}\Longrightarrow w_n=v.$ 
If $n=0$ then $u=v$. The same way we can define the relation $u \overset{+}{\Longrightarrow} v$ except that  $n\ge 1$ always, so at least one direct derivation will de used.   

\vspace{-2mm}
\begin{defi}
The \ki{language generated}\inddef{language generated} by grammar $G=(N,T,P,S)$ is the set  

\[L(G) = \{ u\in T^* \; |\; S \overset{*}{\Longrightarrow} u \} \koz .\] 
\end{defi}

\noindent So $L(G)$ contains  all words over the alphabet $T$ which can be derived  
from the start symbol $S$ using the productions from $P$. 

\begin{pelda} 
Let $G=(N,T,P,S)$ where 

$N= \{S\},$ 

$T=\{a, b\},$ 

$P= \{S\rightarrow aSb, \; S\rightarrow ab \}.$ 

\noindent It is easy to see than $L(G)= \{a^nb^n\; |\; n\ge 1  \}$ because
\[ S \underset{G}\Longrightarrow aSb \underset{G}\Longrightarrow a^2Sb^2 
\underset{G}\Longrightarrow \cdots \underset{G}\Longrightarrow a^{n-1}Sb^{n-1}\underset{G}\Longrightarrow
  a^nb^n \koz ,
\]
where up to the last but one replacement the first production 
 ($S\rightarrow aSb$) was used, while at the last replacement the production $S\rightarrow ab$. This derivation can be written
$S \overset{*}{\underset{G} \Longrightarrow} a^nb^n.$ 
 Therefore $a^nb^n$  can be derived from $S$ for all $n$ and no other words can be derived from 
  $S$. 
\end{pelda}

\vspace{-4mm}
\begin{defi}
Two grammars $G_1$ and $G_2$ are \ki{equivalent,} and this is denoted by
$G_1\cong G_2$ if  $L(G_1)=L(G_2)$. 
\end{defi}

\vspace{-2mm}
\begin{pelda} \label{KZoldal} The following two grammars are equivalent because both of them generate the language 
$\{a^nb^nc^n \; |\; n\ge 1\}$.

\noindent $G_1= (N_1, T, P_1, S_1)$, where

$N_1= \{ S_1, X,Y \} $, $ \; T=\{a,b,c\},$ 

$P_1 = \{ S_1\rightarrow abc, \; S_1\rightarrow aXbc, \; Xb \rightarrow bX, \; Xc \rightarrow Ybcc, \; 
 bY\rightarrow Yb, \ aY\rightarrow aaX, \; aY\rightarrow aa \}.$

\noindent $G_2=(N_2, T, P_2, S_2),$ where

$N_2=\{ S_2, A, B, C\},$

$P_2 = \{ S_2 \rightarrow aS_2BC,\; S_2\rightarrow aBC, \; CB\rightarrow BC, \; aB\rightarrow ab, \;
bB\rightarrow bb, \ bC \rightarrow bc, \; cC\rightarrow cc \}.$ 

\noindent
First let us prove by mathematical induction that for   
$n\ge 2$   $S_1 \overset{*}{\underset{G_1}\Longrightarrow} a^{n-1}Yb^nc^n.$ If $n=2$ then 
\[
S_1\underset{G_1}\Longrightarrow aXbc\underset{G_1}\Longrightarrow abXc\underset{G_1}\Longrightarrow abYbcc
\underset{G_1}\Longrightarrow aYb^2c^2 \koz .
\]
The inductive hypothesis is  $S_1 \overset{*}{\underset{G_1}\Longrightarrow} a^{n-2}Yb^{n-1}c^{n-1}.$
We use production $aY\rightarrow aaX$, then $(n-1)$ times  production $Xb\rightarrow bX$, and then 
 production $Xc\rightarrow Ybcc$, afterwards again $(n-1)$ times production
$bY\rightarrow Yb$. Therefore
\[ S_1\underset{G_1}\Longrightarrow a^{n-2}Yb^{n-1}c^{n-1}\underset{G_1}\Longrightarrow a^{n-1}Xb^{n-1}c^{n-1} 
\overset{*}{\underset{G_1}\Longrightarrow} \]
\[ a^{n-1}b^{n-1}Xc^{n-1} \underset{G_1}\Longrightarrow  a^{n-1}b^{n-1}Ybc^n 
\overset{*}{\underset{G_1}\Longrightarrow} a^{n-1}Yb^nc^n \koz . \]

\newpage
If now we use production $aY\rightarrow aa$ we get 
$S_1 \overset{*}{\underset{G_1}\Longrightarrow} a^nb^nc^n$ for $n\ge 2$, but 
$S_1\underset{G_1}\Longrightarrow abc$ by the production  $S_1\rightarrow abc$, so $a^nb^nc^n\in L(G_1)$ for any $n\ge 1$.
We have to prove in addition that using the productions of the grammar we cannot derive only words of the form
 $a^nb^nc^n$. It is easy to see that a successful derivation  
 (which ends in a word containing only terminals) can be obtained only in the presented way.

\noindent Similarly  for $n\ge 2$
\[ S_2\underset{G_2}\Longrightarrow aS_2BC \overset{*}{\underset{G_2}\Longrightarrow} a^{n-1}S_2(BC)^{n-1} 
\underset{G_2}\Longrightarrow a^n(BC)^n  \overset{*}{\underset{G_2}\Longrightarrow} a^nB^nC^n \]

\[\quad\; 
\underset{G_2}\Longrightarrow a^nbB^{n-1}C^n\overset{*}{\underset{G_2}\Longrightarrow} a^nb^nC^n
\underset{G_2}\Longrightarrow a^nb^ncC^{n-1}\overset{*}{\underset{G_2}\Longrightarrow} a^nb^nc^n \koz .
 \]
Here orderly were used the productions  
 $S_2\rightarrow aS_2BC$ ($n-1$ times), $S_2\rightarrow aBC$, $CB\rightarrow BC$ ($n-1$ times),
$aB\rightarrow ab$, $bB\rightarrow bb$ ($n-1$ times), $bC\rightarrow bc$, $cC\rightarrow cc$ ($n-1$ times).
But $S_2\underset{G_2}\Longrightarrow aBC 
\underset{G_2}\Longrightarrow abC\underset{G_2}\Longrightarrow abc,$  
So $S_2\overset{*}{\underset{G_2}\Longrightarrow} 
a^nb^nc^n$, $n\ge 1$. It is also easy to see than other words cannot be derived using grammar $G_2$.

\medskip
\noindent The grammars 

$G_3=( \{S\}, \{a,b\}, \{ S\rightarrow aSb, \; S\rightarrow  \varepsilon\} , S)$
 and
 
$G_4=( \{S\}, \{a,b\}, \{ S\rightarrow aSb, \; S\rightarrow ab \} , S)$ \\
are not equivalent because $L(G_3)\setminus \{\varepsilon\} = L(G_4)$. 
\end{pelda}

\begin{tetel} Not all languages can be generated by grammars.
\end{tetel}
\begin{biz}
We encode grammars for the proof as words on the alphabet $\{0,1\}$. For a given grammar $G=(N,T,P,S)$ 
 let $N=\{S_1,S_2,\ldots, S_n\},$  $T=\{a_1,a_2,\ldots, a_m\}$ and  $S=S_1.$ 
The encoding is the following:

\smallskip
the code of $S_i$ is $10\underbrace{11\ldots 11}_{i\textrm{ times}}01,$ \quad\qquad
the code of $a_i$ is $ 100\underbrace{11\ldots 11}_{i\textrm{ times}}001 \koz . $\\
In the code of the grammar the letters are separated by 000, the code of the arrow is 0000, 
and the productions are separated by 00000.

It is enough, of course, to encode the productions only. For example, consider the grammar 

$G=(\{S\}, \{a,b\},  \{S\rightarrow aSb, \; S\rightarrow ab\} , S)$.
\\ The code of $S$ is 10101, the code of $a$ is 1001001, the code of $b$ is 10011001. 
 The code of the grammar is

\smallskip
$\underbrace{10101}0000\underbrace{1001001}000\underbrace{10101}000
\underbrace{10011001}00000\underbrace{10101}0000\underbrace{1001001}000$\\ 

$\underbrace{10011001} \koz .$

\smallskip
From this encoding results that the grammars with terminal alphabet $T$ can be enumerated 
\footnote{ Let us suppose that in the alphabet $\{ 0,1 \}$ there is a linear order $<$, let us say $0<1$. 
The words which are  codes of grammars can be enumerated by ordering them first after their lengths, 
and inside the equal length words, alphabetically, using the order of their letters. 
But we can use equally the lexicographic order, which means that $u<v$ ($u$ is before $v$) if  
$u$ is a proper prefix of  $v$ or there exists the decompositions  $u=xay$ and $v=xby'$, 
where  $x$, $y$, $y'$ are subwords, $a$ and  $b$ letters with  $a<b$.} as 
$G_1, G_2, \ldots , G_k, \ldots \ ,$   and the set of these grammars is a denumerable infinite set.

Consider now the set of all languages over $T$ denoted by  
${\cal L}_T=\{ L\;|\; L\subseteq T^*\}$, that is ${\cal L}_T={\cal P}(T^*)$. 
The set  $T^*$ is denumerable because its words can be ordered.  Let this order 
$s_0, s_1, s_2, \ldots $, where $s_0=\varepsilon$. We associate to each language  
$L\in  {\cal L}_T$  an infinite binary sequence $b_0, b_1, b_2, \ldots$ 
the following way:
\[  b_i= \left\{ 
             \begin{array}{ll}
                  1,  & \textrm{ if }\; s_i\in L\\
                  0,  & \textrm{ if }\; s_i\not\in L\\
             \end{array}  
        \right.  
                 \qquad i=0,1,2,\ldots \koz .
\] 
It is easy to see that the set of all such binary sequences is not denumerable, because each sequence can be considered as a positive number less than 1 using its binary representation (The decimal point is considered to be before the first digit). Conversely, to each positive number less than 1 in binary representation  a binary sequence can be associated. So, the cardinality of the set of infinite binary sequences is equal to cardinality of interval $[0,1]$, which is of continuum power.  Therefore the set ${\cal L}_T$ is of continuum cardinality.  Now to each grammar with terminal alphabet $T$ associate the corresponding generated language over
$T$. Since the cardinality of the set of grammars is denumerable, there will exist a language from 
${\cal L}_T$, without associated grammar, a language which cannot be generated by a grammar.  
\end{biz}

\subsection{Chomsky hierarchy of grammars and languages}\index{Chomsky hierarchy}\nevindex{Chomsky, Noam} 

Putting some restrictions on the form of productions, four type of grammars can be distinguished.

\begin{defi}
Define for a grammar $G=(N,T,P,S)$ the following four types.

A grammar $G$ is of type 0 (\ki{phrase-structure grammar}) if there are no restrictions on productions. 
 \inddef{grammar!phrase structure}

A grammar $G$ is of type 1 \ki{(context-sensitive grammar)} if all of its productions are of the form $\alpha A \gamma \rightarrow \alpha \beta\gamma$, \inddef{grammar!context-sensitive} 
 where  $A\in N$, $\alpha, \gamma\in (N\cup T)^*$, $\beta\in (N\cup T)^+$. 
 A production of the form  $S\rightarrow \varepsilon$ can also be accepted if  the start symbol $S$ does not occur in the right-hand side of any production. 

A grammar $G$ is of type  2 \ki{(context-free grammar)} if all of its productions   \inddef{grammar!context-free} are of the form $A\rightarrow \beta$, where $A\in N$, $\beta\in (N\cup T)^+$. 
 A production of the form  $S\rightarrow \varepsilon$ can also be accepted if  the start symbol $S$ does not occur in the right-hand side of any production. 

A grammar $G$ is of type 3  \ki{(regular grammar)} if its productions are of the form  $A\rightarrow aB$ or $A\rightarrow a$, where \inddef{grammar!regular}
 $a\in T$ and $A,B\in N$.
A production of the form  $S\rightarrow \varepsilon$ can also be accepted if  the start symbol $S$ does not occur in the right-hand side of any production. 

If a grammar $G$ is of type $i$ then language $L(G)$ is also of type $i$.  
\end{defi}

This classification was introduced by Noam Chomsky.

\index{language!null@of type 0,1,2,3}
\index{language!regular}
\index{language!context-free}
\index{language!context-sensitive}
\index{language!phrase-structure}
\index{grammar!null@of type 0,1,2,3}

A language  $L$ is of type  $i$  ($i=0,1,2,3$) if there exists a grammar $G$ of type  $i$  which generates the language $L$, so $L=L(G)$.  

Denote by ${\cal L}_i$ ($i=0,1,2,3$) the class of the languages of type $i$. Can be proved that 
\[ {\cal L}_0 \supset {\cal L}_1 \supset {\cal L}_2 \supset {\cal L}_3 \koz .\] 
By the definition of different type of languages, the inclusions  ($\supseteq$) are evident, but the strict inclusions  ($\supset$) must be proved. 

\begin{pelda} We give an example for each type of context-sensitive, context-free and regular grammars.

\noindent \textit{Context-sensitive grammar.}
$G_1=(N_1,T_1,P_1,S_1)$, where $N_1=\{S_1, A, B, C\}, \quad T_1= \{ a, 0,1\}.$

Elements of $P_1$ are:

$\begin{array}{lll}
 S_1  &\rightarrow&  ACA, \\
 AC   &\rightarrow& AACA\; |\; ABa \;|\;AaB,\\
 B    &\rightarrow& AB\;|\;A,\\
 A    &\rightarrow& 0\;|\;1.
\end{array} $

 Language $L(G_1)$ contains words of the form $uav$ with $u,v\in \{0,1\}^*$ and 
$ |u| \not= |v|$.

\noindent \textit{Context-free grammar.}
$G_2=(N_2,T_2,P_2,S)$, where $N_2=\{S, A,B\}, \quad T_2= \{ +,*,(,),a\}.$

Elements of $P_2$ are:

$\begin{array}{lll}
 S   &\rightarrow& S+A\;|\; A,\\
 A   &\rightarrow& A*B \;|\; B,\\
 B   &\rightarrow& (S) \;|\; a.
\end{array}$

Language $L(G_2)$ contains algebraic expressions which can be correctly built using letter $a$, operators  $+$ and $*$ and brackets.   

\noindent \textit{Regular grammar.}
$G_3=(N_3,T_3,P_3,S_3)$, where $N_3=\{S_3, A, B\}, \quad T_3= \{a,b\}.$

Elements of $P_3$ are:

$\begin{array}{lll}
 S_3 &\rightarrow& aA\\
 A   &\rightarrow& aB\;|\;a\\
 B   &\rightarrow& aB\;|\; bB\;|\; a\;|\;b .
\end{array}$

Language $L(G_3)$ contains words over the alphabet $\{a,b\}$ with at least two letters  $a$ at the beginning.
\end{pelda}


It is easy to prove that any finite language is regular. 
The productions will be done to generate all words of the language. 
For example, if $u=a_1a_2\ldots a_n$ is in the language, 
then we introduce the productions: $S\rightarrow a_1A_1$, $A_1\rightarrow a_2A_2$, \ldots 
$A_{n-2}\rightarrow a_{n-1}A_{n-1} $, $A_{n-1}\rightarrow a_n$, where $S$ is the start 
symbol of the language and  $A_1, \ldots, A_{n-1}$ are distinct nonterminals. 
We define such productions for all words of the language using different 
nonterminals for different words, excepting the start symbol $S$. 
If the empty word is also an element of the language, then the production  
$S\rightarrow \varepsilon$ is also considered.

The empty set is also a regular language, because the regular grammar 
$G=(\{S\}, \{a\}, \{S\rightarrow aS\} , S)$ generates it.

\subsubsection{Eliminating unit productions.}
A production of the form  $A\rightarrow B$ is called a \ki{unit production,}  
where $A,B\in N$.\inddef{unit production} Unit productions can be 
eliminated from a grammar in such a way that the new grammar 
will be of the same type and equivalent to the first one.

Let $G=(N,T,P,S)$ be a grammar with unit productions. Define an equivalent grammar  
 $G'=(N,T, P', S)$ without unit productions. The following algorithm will construct the equivalent grammar.

\begin{algN}{Eliminate-Unit-Productions$(G)$}\inddef{Eliminate-Unit-Production@\textsc{Eliminate-Unit-Productions}} \label{KZatnevezes}
1 \' if the unit productions $A \rightarrow B$ and $B\rightarrow C$ are in  $P$ put  also      \\
  \' the unit production $A\rightarrow C$ in $P$ while $P$ can be extended,\\
2 \' if the unit production $A \rightarrow B$ and the production 
$B\rightarrow \alpha$ ($\alpha\not\in N$) are in  $P$\\
 \' put also the production  $A\rightarrow \alpha$ in $P$,\\
3 \' let $P'$ be the set of productions of $P$ except unit productions \\
4 \' \key{return} $G'$
\end{algN}

Clearly,  $G$ and $G'$ are equivalent. If  $G$ is of type $i\in \{0,1,2,3\}$ then  $ G'$ is also of type $i.$ 

\begin{pelda}
Use the above algorithm in the case of the grammar $G=\big( \{S,A,B,C\}, \{a,b\}, P, S \big)$, where $P$ contains

$\begin{array}{lllllllll}
   S\rightarrow A,  &\quad& A\rightarrow B,  &\quad& B\rightarrow C, &\quad& C\rightarrow B,  &\quad& D\rightarrow C,\\
   S\rightarrow B,  && A\rightarrow D,  &&                && C\rightarrow Aa, && \\
                   && A\rightarrow aB, &&                &&                 &&  \\
                   && A \rightarrow b. &&                &&                 &&
\end{array} $

\noindent Using the first step of the algorithm, we get the following new unit productions:

 $S\rightarrow D$ \qquad\quad  (because of $S\rightarrow A\;$ and $\;A\rightarrow D$),

 $S\rightarrow C$ \qquad\quad (because of $S\rightarrow B\;$ and $\;B\rightarrow C$),

 $A\rightarrow C$ \qquad\quad (because of $A\rightarrow B\;$ and $\;B\rightarrow C$),

 $B\rightarrow B$ \qquad\quad (because of $B\rightarrow C\;$ and $\;C\rightarrow B$), 

 $C\rightarrow C$ \qquad\quad (because of $C\rightarrow B\;$ and $\;B\rightarrow C$), 

 $D\rightarrow B$ \qquad\quad (because of $D\rightarrow C\;$ and $\;C\rightarrow B$).

\noindent In the second step of the algorithm will be considered only productions with  
$A$ or $C$ in the right-hand side, since productions  $A\rightarrow aB$, 
$A\rightarrow b$ and  $C\rightarrow Aa$ can be used (the other productions are all 
unit productions). We get the following new productions:

$S\rightarrow aB$ \qquad\quad  (because of $S\rightarrow A\;$  and $\;A\rightarrow aB$),
 
$S\rightarrow b$ \qquad\quad\;\; (because of $S\rightarrow A\;$ and $\;A\rightarrow b$),

$S\rightarrow Aa$ \qquad\quad (because of $S\rightarrow C\;$ and $\;C\rightarrow Aa$),

$A\rightarrow Aa$ \qquad\quad (because of $A\rightarrow C\;$ and $\;C\rightarrow Aa$),

$B\rightarrow Aa$ \qquad\quad (because of $B\rightarrow C\;$ and $\;C\rightarrow Aa$).

\noindent The new grammar $G'=\big( \{S,A,B,C\}, \{a,b\}, P', S \big)$ will have the productions:

$\begin{array}{lllllll}
   S\rightarrow b,   &\quad& A\rightarrow b,   &\quad& B\rightarrow Aa, &\quad& C\rightarrow Aa,  \\
   S\rightarrow aB,  &     & A\rightarrow aB,  &&                       &&   \\
   S\rightarrow Aa   &     & A\rightarrow Aa,  &&                       &&                 
\end{array} $
\end{pelda}

\subsubsection{Grammars in normal forms.} 

\index{grammar!normal form}\label{normalalak}
A grammar is to be said a \ki{grammar in normal form} if its productions have no terminal symbols in the left-hand side. 

We need the following notions. For alphabets $\Sigma_1$ and  $\Sigma_2$ a \emph{homomorphism} is a function  $h: \Sigma_1^*\rightarrow \Sigma_2^*$ for which $h(u_1u_2)=h(u_1)h(u_2), \; \forall u_1,u_2\in \Sigma_1^*$.  
It is easy to see that for arbitrary $u=a_1a_2\ldots a_n\in \Sigma_1^*$ value $h(u)$ is uniquely determined  by the restriction of $h$ on $\Sigma_1$, because $h(u)=h(a_1)h(a_2)\ldots h(a_n).$

If a homomorphism $h$ is a bijection then $h$ is an \emph{isomorphism}.

\begin{tetel}
To any grammar an equivalent grammar in normal form can be associated.
\end{tetel}
\begin{biz}
Grammars of type 2 and 3 have in left-hand side of any productions only a nonterminal, so they are in normal form. 
The proof has to be done for grammars of type 0 and 1 only.

Let $G=(N,T, P,S)$ be the original grammar and we define the grammar in normal form as 
$G'=(N', T, P', S)$.

Let $a_1, a_2, \ldots , a_k$ be those terminal symbols which occur in the left-hand side of productions. 
We introduce the new nonterminals $A_1,A_2, \ldots, A_k$. The following notation will be used: 
$T_1=\{a_1, a_2, \ldots , a_k\}$, $T_2=T\setminus T_1$, $N_1=\{A_1,A_2, \ldots, A_k\}$
and  $N'=N\cup N_1$.

Define the isomorphism  $h: N\cup T \longrightarrow  N'\cup T_2$, where 
\begin{tabbing}
\qquad $h(a_i)=A_i$,\quad  \= if \quad $a_i\in T_1$, \\
\qquad $h(X)=X$,     \> if \quad  $X\in N\cup T_2.$
\end{tabbing}
 
Define the set  $P'$ of production as 
\[ P'= \Big\{ h(\alpha)\rightarrow h(\beta) \;\big|\;(\alpha\rightarrow \beta)\in P  \Big\} \cup 
\Big\{ A_i\longrightarrow a_i\;\big|\; i=1,2, \ldots, k\Big\} \koz .\]

In this case $\alpha \overset{*}{\underset{G}{\Longrightarrow}}\beta $ if and only if 
$h(\alpha) \overset{*}{\underset{G'}{\Longrightarrow}}h(\beta).$ From this the theorem immediately results because
 $S\overset{*}{\underset{G}{\Longrightarrow}}u \; 
\Leftrightarrow \; S=h(S) \overset{*}{\underset{G'}{\Longrightarrow}}h(u) = u.$
\end{biz}

\begin{pelda}
Let $G=(\{S,D,E\},\{a,b,c,d,e\}, P, S)$, where $P$ contains

$\begin{array}{lll}
S  & \rightarrow & aebc \;|\; aDbc\\
Db & \rightarrow & bD\\
Dc & \rightarrow & Ebccd\\
bE & \rightarrow & Eb\\
aE & \rightarrow & aaD \;|\; aae.
\end{array}$

In the left-hand side of productions  the terminals  $a,b,c$ occur, therefore consider the new nonterminals  $A,B,C$, and include in $P'$ also the new productions  $A\rightarrow a$, $B\rightarrow b$ and $C\rightarrow c$.

Terminals  $a, b,c$ will be replaced  by nonterminals $A,B,C$  respectively, and we get the set $P'$ as

$\begin{array}{lll}
S  & \rightarrow & AeBC \;|\; ADBC\\
DB & \rightarrow & BD\\
DC & \rightarrow & EBCCd\\
BE & \rightarrow & EB\\
AE & \rightarrow & AAD \;|\; AAe\\
A  & \rightarrow & a\\
B  & \rightarrow & b\\
C  & \rightarrow & c.\\
\end{array}$

Let us see what words can be generated by this grammars. It is easy to see that  $aebc\in L(G'),$ because $S\Longrightarrow AeBC\overset{*}{\Longrightarrow}aebc$.

$S\Longrightarrow ADBC \Longrightarrow ABDC\Longrightarrow ABEBCCd\Longrightarrow AEBBCCd\Longrightarrow AAeBBCCd \overset{*}{\Longrightarrow} aaebbccd$, so $aaebbccd \in L(G').$  

We prove, using the mathematical induction, that  $S\overset{*}{\Longrightarrow} A^{n-1}EB^nC(Cd)^{n-1}$ for $n\ge 2$. For  $n=2$ this is the case, as we have seen before. 
Continuing the derivation we get
$S\overset{*}{\Longrightarrow} A^{n-1}EB^nC(Cd)^{n-1}\Longrightarrow A^{n-2}AADB^nC(Cd)^{n-1}\overset{*}{\Longrightarrow}
A^nB^nDC(Cd)^{n-1} \Longrightarrow A^nB^nEBCCd(Cd)^{n-1} \overset{*}{\Longrightarrow} A^nEB^{n+1}CCd(Cd)^{n-1}=A^nEB^{n+1}C(Cd)^n$, and this is what we had to prove. 

But $S\overset{*}{\Longrightarrow} A^{n-1}EB^nC(Cd)^{n-1}\Longrightarrow 
A^{n-2}AAeB^nC(Cd)^{n-1}\overset{*}{\Longrightarrow} a^neb^nc(cd)^{n-1}.$  
So $ a^neb^nc(cd)^{n-1}\in L(G'),\; n\ge 1$. These words can be generated also in  $G$.
\end{pelda}

\subsection{Extended grammars}
In this subsection extended grammars of type 1, 2 and 3 will be presented.

\ki{Extended grammar of type 1.}\inddef{extended grammar}
All productions are of the form $\alpha\rightarrow \beta$, where $|\alpha|\le |\beta|$, excepted possibly the production  $S\rightarrow \varepsilon$.

\ki{Extended grammar of type 2.}
All productions are of the form $A\rightarrow \beta$, where $A\in N, \beta \in (N\cup T)^*.$

\ki{Extended grammar of type 3.}
All productions are of the form  $A\rightarrow uB$ or $A\rightarrow u$, Where $A,B\in N, u\in T^*$.
 
\begin{tetel}
To any extended grammar an equivalent grammar of the same type can be associated. 
\end{tetel}
\begin{biz}
Denote by $G_{ext}$ the extended grammar and by  $G$ the corresponding equivalent grammar of the same type. 

\textit{Type 1.} Define the productions of grammar $G$ by rewriting the productions $\alpha\rightarrow \beta$, where $|\alpha|\le |\beta|$ of the extended grammar $G_{ext}$ in the form  $\gamma_1\delta\gamma_2\rightarrow \gamma_1\gamma\gamma_2$  allowed in the case of grammar $G$ by the following way. 

Let  $X_1X_2\ldots X_m\rightarrow Y_1Y_2\ldots Y_n $ ($m\le n$) be a production of   $G_{ext}$, which is not in the required form. Add to the set of productions of $G$ the following productions,  
where $A_1, \ A_2, \ \ldots, \ A_m$ are new nonterminals:

$\begin{array}{lll}
X_1X_2\ldots X_m&\rightarrow&A_1X_2X_3\ldots X_m\\ 
A_1X_2\ldots X_m&\rightarrow& A_1A_2X_3\ldots X_m\\
 &\ldots&\\ 
A_1A_2\ldots A_{m-1}X_m&\rightarrow& A_1A_2\ldots A_{m-1}A_m\\
A_1A_2\ldots A_{m-1}A_m&\rightarrow& Y_1A_2\ldots A_{m-1}A_m\\
Y_1A_2\ldots A_{m-1}A_m&\rightarrow& Y_1Y_2\ldots A_{m-1}A_m\\
 &\ldots&\\
Y_1Y_2\ldots Y_{m-2}A_{m-1}A_m&\rightarrow& Y_1Y_2\ldots Y_{m-2}Y_{m-1}A_m\\
Y_1Y_2\ldots Y_{m-1}A_m&\rightarrow& Y_1Y_2\ldots Y_{m-1}Y_mY_{m+1}\ldots Y_n .
\end{array}$

Furthermore, add to the set of productions of $G$ without any modification the productions of $G_{ext}$ which are of permitted form, i.e. $\gamma_1\delta\gamma_2\rightarrow \gamma_1\gamma\gamma_2$. 
 
Inclusion  $L(G_{ext})\subseteq L(G)$ can be proved because each used production of $G_{ext}$ in a derivation can be simulated by productions $G$ obtained from it. Furthermore, since the productions of $G$ can be used only in the prescribed order, we could not obtain other words, so $L(G)\subseteq L(G_{ext})$ also is true.

\textit{Type 2.} Let $G_{ext}=(N, T, P, S)$. Productions of form $A\rightarrow \varepsilon$ have to be eliminated, only $S\rightarrow \varepsilon$ can remain, if $S$ doesn't occur in the right-hand side of productions.
For this define the following sets:

$U_0=\{A\in N\;|\; (A\rightarrow \varepsilon)\in P\}$

$U_i=U_{i-1}\cup \{ A\in N\;|\; (A\rightarrow w)\in P, \; w\in U_{i-1}^+\} .$

Since for $i\ge 1$ we have $U_{i-1}\subseteq U_i$, $U_i\subseteq N$ and $N$ is a finite set, there must exists such a $k$ for which  $U_{k-1}=U_k$. Let us denote this set as $U$. It is easy to see that a nonterminal $A$ is in $U$ if and only if $A\overset{*}{\Longrightarrow}\varepsilon$. (In addition  $\varepsilon \in L(G_{ext})$ if and only if $S\in U$.)

We define the productions of $G$ starting from the productions of $G_{ext}$ in the following way. For each production  $A\rightarrow \alpha$ with $\alpha \not= \varepsilon$ of $G_{ext}$ add to the set of productions of  $G$ this one and all productions which can be obtained from it by eliminating from $\alpha$ one or more nonterminals which are in $U$, but only in the case when the right-hand side does not become $\varepsilon$. 

It in not difficult to see that this grammar $G$ generates the same language as  $G_{ext}$ does, except the empty word  $\varepsilon$. So, if $\varepsilon \not\in L(G_{ext})$ then the proof is finished.  But if $\varepsilon\in L(G_{ext})$, then there are two cases.  If the start symbol $S$ does not occur in any right-hand side of productions, then by introducing the production $S\rightarrow \varepsilon$, grammar $G$ will generate also the empty word. If $S$ occurs in a production in the right-hand side, then we introduce a new start symbol $S'$ and the new productions $S'\rightarrow S$ and $S'\rightarrow \varepsilon$. Now the empty word  $\varepsilon$ can also be generated by grammar  $G$. 
 
\textit{Type 3.} First we use for $G_{ext}$ the procedure defined for grammars of type 2 to eliminate productions of the form  $A\rightarrow \varepsilon$. From the obtained grammar we eliminate the unit productions 
using the algorithm \textsc{Eliminate-Unit-Productions} (see page \pageref{KZatnevezes}).

In the obtained grammar for each production  $A\rightarrow a_1a_2\ldots a_nB$, where
$B\in N\cup \{\varepsilon\}$, add to the productions of $G$ also the followings

$\begin{array}{lll}
A  &\rightarrow& a_1A_1,\\
A_1 &\rightarrow&  a_2A_2,\\
    &\ldots& \\
A_{n-1}&\rightarrow&  a_n B,
\end{array}$

\noindent where $A_1, A_2, \ldots, A_{n-1}$ are new nonterminals. 
It is easy to prove that grammar $G$  built in this way is equivalent to $G_{ext}$.
\end{biz}

\begin{pelda} Let $G_{ext}=(N,T, P,S)$ be an extended grammar of type 1, where 
$N=\{S,B,C\}$, $T=\{a,b,c\}$ and $P$ contains the following productions: 

$\begin{array}{llllll}
S &\rightarrow& aSBC\;|\; aBC  \quad\qquad &  CB &\rightarrow& BC\\
aB &\rightarrow& ab                        &  bB &\rightarrow &bb\\
bC &\rightarrow &bc                        &  cC &\rightarrow  cc \koz .
\end{array}$

\noindent The only production which is not context-sensitive is $CB\rightarrow BC$. Using the method given in the proof, 
we introduce the productions: 

$\begin{array}{lll}
CB &\rightarrow &AB\\
AB &\rightarrow &AD\\
AD &\rightarrow &BD\\
BD &\rightarrow & BC
\end{array} $

\noindent Now the grammar $G=(\{S,A,B,C,D\}, \{a,b,c\}, P', S)$ is context-sensitive, where the elements of  $P'$ are

$\begin{array}{llllll}
S & \rightarrow & aSBC\;|\; aBC\quad \qquad &&&\\
 CB  &\rightarrow&   AB &  aB  &\rightarrow&   ab\\
 AB  &\rightarrow&   AD &  bB  &\rightarrow&   bb\\
 AD  &\rightarrow&   BD &  bC  &\rightarrow&   bc\\
 BD  &\rightarrow&   BC &  cC  &\rightarrow&  cc.
\end{array}$

\noindent It can be proved that $L(G_{ext})=L(G)=\{a^nb^nc^n\;|\; n\ge 1\}$.
\end{pelda}


\begin{pelda} Let $G_{ext}=(\{S,B,C\}, \{a,b,c\}, P, S)$ be an extended grammar of type 2, where $P$ contains:

$\begin{array}{lll}
 S&\rightarrow& aSc\;  |\; B\\
 B&\rightarrow& bB \;  |\; C\\
 C&\rightarrow& Cc  \; |\; \varepsilon.
\end{array}$

\noindent Then $U_0=\{C\}$, $U_1=\{B,C\}$, $U_3=\{S,B,C\}=U$. The productions of the new grammar are:

$\begin{array}{lll}
S&\rightarrow& aSc\;|\; ac \;|\; B\\
B&\rightarrow& bB\;|\; b\;|\; C\\
C&\rightarrow& Cc\;|\; c.
\end{array}$

\noindent The original grammar generates also the empty word and because $S$ occurs in the right-hand side of a production, a new start symbol and two new productions will be defined:  $S'\rightarrow S,  S'\rightarrow \varepsilon$. The context-free grammar equivalent to the original grammar is 
$G=(\{S',S,B,C\}, \{a,b,c\}, P', S')$ with the productions: 

$\begin{array}{lll}
 S'&\rightarrow& S\;|\; \varepsilon\\
 S&\rightarrow& aSc\;|\; ac \;|\; B\\
 B&\rightarrow& bB\;|\; b\;|\; C\\
 C&\rightarrow& Cc\;|\; c.
\end{array}$

\noindent Both of these grammars generate language $\{a^mb^nc^p\;|\; p\ge m\ge 0, n\ge 0\}$.
\end{pelda}
\begin{pelda}
 Let $G_{ext}=(\{S,A,B\}, \{a,b\}, P,S)$ be the extended grammar of type  3 under examination, where $P$:

$\begin{array}{lll}
S&\rightarrow& abA \\
A&\rightarrow& bB\\
B&\rightarrow& S \;|\; \varepsilon.
\end{array}$

\noindent First, we eliminate production $B\rightarrow \varepsilon$. Since $U_0=U=\{B\}$,  the productions will be 

$\begin{array}{lll}
S&\rightarrow& abA\\
A&\rightarrow& bB\;|\; b\\
B&\rightarrow& S.
\end{array}$

\noindent The latter production (which a unit production) can also be eliminated, by replacing 
it with $B\rightarrow abA$. Productions $S\rightarrow abA$ and $B\rightarrow abA$ have to be transformed. 
Since, both productions have the same right-hand side, it is enough to introduce only one new 
nonterminal and to use the productions $S\rightarrow aC$  and  $C\rightarrow bA$ 
instead of $S\rightarrow abA$. Production $B\rightarrow abA$ will be replaced by  
$B\rightarrow aC$. The new grammar is $G=(\{S,A,B,C\}, \{a,b\}, P', S)$, where $P'$:

$\begin{array}{lll}
S&\rightarrow& aC\\
A&\rightarrow& bB\;|\; b\\
B&\rightarrow& aC\\
C&\rightarrow& bA.
\end{array}$

\noindent Can be proved that $L(G_{ext})=L(G)=\{(abb)^n\;|\; n\ge 1\}$.
\end{pelda}

\subsection{Closure properties in the Chomsky-classes}
We will prove the following theorem, by which  the Chomsky-classes of languages  are closed under the regular operations, that is, the union and product of two languages of type $i$ is also of type  $i$, the iteration of a language of type $i$ is also of type  $i$  ($i=0,1,2,3$). 
\begin{tetel}\label{KZzartsagi}
The class  ${\mathcal L}_i$ ($i=0,1,2,3$) of languages is closed under the regular operations.
\end{tetel}
\begin{biz}
For the proof we will use extended grammars. Consider the extended grammars $G_1=(N_1, T_1, P_1, S_1)$ and $G_2=(N_2,T_2,P_2,S_2)$ of type $i$ each. We can suppose that $N_1\cap N_2=\emptyset$.

\textit{Union.} 
Let $G_\cup=(N_1\cup N_2\cup \{S\}, T_1\cup T_2, 
            P_1\cup P_2\cup \{ S\rightarrow S_1, S\rightarrow S_2\}, S)$.

We will show that $L(G_\cup)=L(G_1)\cup L(G_2).$
If $i=0,2,3$ then from the assumption that $G_1$ and  $G_2$ are of type $i$ follows by definition that $G_\cup$ also is of type $i$. If $i=1$ and one of the grammars generates the empty word, then we eliminate from $G_\cup$ the corresponding production (possibly the both) $S_k\rightarrow \varepsilon$ ($k=1,2$) and replace it by  production 
$S\rightarrow \varepsilon$.

\textit{Product.} 
Let $G_{\times}=(N_1\cup N_2\cup \{S\}, T_1\cup T_2, 
            P_1\cup P_2\cup \{ S\rightarrow S_1S_2\}, S)$.

We will show that $L(G_{\times})=L(G_1)L(G_2).$
By definition, if $i=0,2$ then $G_{\times}$ will be of the same type.
If $i=1$ and there is production $S_1\rightarrow \varepsilon$ in  $P_1$ but  there is no production $S_2\rightarrow \varepsilon$ in  $P_2$ then production $S_1\rightarrow \varepsilon$ will be replaced by $S\rightarrow S_2$. We will proceed the same way in the symmetrical case.  
 If there is in  $P_1$ production   $S_1\rightarrow \varepsilon$ 
and in  $P_2$ production  $S_2\rightarrow \varepsilon$ then they will be replaced by  $S\rightarrow \varepsilon$.

In the case of regular grammars  ($i=3$), because  $S\rightarrow S_1S_2$ is not a regular production, 
we need to use another grammar $G_{\times}=(N_1\cup N_2, T_1\cup T_2, 
            P_1'\cup P_2, S_1)$, where  the difference between $P_1'$ and  $P_1$ lies 
			in that instead of productions in the form $A\rightarrow u, u\in T^*$ in $P_1'$ 
			will exist production of the form $A\rightarrow uS_2$.

\textit{Iteration.} 
Let $G_{\textstyle *}=(N_1\cup \{S\}, T_1, P, S)$.

In the case of grammars of type 2 let $P=P_1\cup \{ S\rightarrow S_1S,\; S\rightarrow \varepsilon\}$. Then  
$G_{\textstyle *}$ also is of type 2. 

In the case of grammars of type  3, as in the case of product, we will change the productions, that is  $P=P'_1\cup \{ S\rightarrow S_1,\; S\rightarrow \varepsilon\}$, where  the difference between
$P'_1$ and   $P_1$ lies in that for each $A\rightarrow u \;(u\in T^*)$ will be replaced by $A\rightarrow uS$, and the others will be not changed. Then  $G_{\textstyle *}$ also will be of type 3. 

The productions given in the case of type 2 are not valid for $i=0,1$, because when applying production $S\rightarrow S_1S$ we can get the derivations of type  $S\overset{*}{\Longrightarrow} S_1S_1$, $\; S_1\overset{*}{\Longrightarrow} \alpha_1\beta_1$,  $\;S_1\overset{*}{\Longrightarrow} \alpha_2\beta_2$, where $\beta_1\alpha_2$ can be a left-hand side of a production. In this case, replacing $\beta_1\alpha_2$ by its right-hand side in derivation $S\overset{*}{\Longrightarrow} \alpha_1\beta_1\alpha_2\beta_2$, we can generate a word which is not in the iterated language. To avoid such situations, first let us assume that the language is in normal form, i.e. the left-hand side of productions does not contain terminals (\emph{see} page \pageref{normalalak}), second we introduce a new nonterminal $S'$, so the set of nonterminals now is $N_1\cup\{S,S'\}$, and the productions are the following:

\centerline{$P=P_1\cup \{S\rightarrow \varepsilon,\; S\rightarrow S_1S' \} \cup \{aS'\rightarrow aS\;|\; 
      a\in T_1\} \koz . $}

\noindent
Now we can avoid situations in which the left-hand side of a production can extend over the limits of words in a derivation because of the iteration.  The above derivations can be used only by beginning with $S\Longrightarrow S_1S'$ and  getting derivation $S\overset{*}{\Longrightarrow} \alpha_1\beta_1S'$. Here we can not replace $S'$ unless the last symbol in $\beta_1$ is a terminal symbol, and only after using a production of the form $aS'\rightarrow aS$. 

It is easy to show that  $L(G_*)=L(G_1)^*$ for each type.type
\end{biz}  

\vspace{-2mm}
\begin{gyak}
\ujgyak
Give a grammar which generates language  $L=\big\{uu^{-1}\;|\; u\in \{ a,b \}^*\big\} $ and determine its type. 
\ujgyak 
Let  $G=(N,T,P,S)$ be an extended context-free grammar, where

$N=\{S,A,C,D\}$, $\;T=\{a,b,c,d,e\},$ 

$P=\{S\rightarrow abCADe, \; C\rightarrow cC, \;  
C\rightarrow \varepsilon,  \; D\rightarrow dD,  \; D\rightarrow \varepsilon,  \; A\rightarrow \varepsilon, \ A\rightarrow dDcCA\}$. 

\noindent Give an equivalent context-free grammar.\gyakindex{grammar!context free}
\ujgyak
Show that  $\Sigma^*$ and  $\Sigma^+$ are regular languages over arbitrary alphabet $\Sigma$.
\ujgyak
Give a grammar to generate language $L=\big\{u\in \{0,1\}^*\;|\;  n_0(u)=n_1(u)  \big\}$, where $n_0(u)$ represents the number of 0's in word  $u$ and  $n_1(u)$  the number of 1's.
\ujgyak
Give a grammar to generate all natural numbers. 
\ujgyak
Give a grammar to generate the following languages, respectively:

$L_1=\{a^nb^mc^p\;|\; n\ge 1, m\ge 1, p\ge 1  \},$ 

$ L_2=\{a^{2n}\;|\; n\ge 1  \}$,

$L_3=\{a^nb^m\;|\; n\ge 0, m\ge 0\; \},$  

$L_4=\{a^nb^m\;|\; n\ge m\ge  1\}$.
\ujgyak 
Let $G=(N,T,P,S)$ be an extended grammar, where $N=\{S,A,B,C\}$, $T=\{a\}$ and  $P$ contains the productions:

\centerline{$S\rightarrow BAB,\; BA\rightarrow BC,\; CA\rightarrow AAC,\; CB\rightarrow AAB,\; 
A\rightarrow a, \;B\rightarrow \varepsilon \koz .$}

\noindent Determine the type of this grammar. Give an equivalent, not extended grammar with the same type. What language it generates?
\end{gyak}


\vspace{-6mm}
\section{Finite automata and regular languages} \inddef{automaton!finite}\index{language!regular}\index{finite automata} \index{regular!language}

Finite automata are computing models with input tape and a finite set of states (Fig. \ref{KZ2ka0}).  Among the states some are called initial and some final. At the beginning the automaton read the first letter of the input word written on the input tape. Beginning with  an initial state, the automaton read the letters of the input word one after another  while change its states, and when after reading the last input letter the current state is a final one, we say that the automaton accepts the given word. The set of words accepted by such an automaton is called the language accepted (recognized) by the automaton. 
 

\begin{defi}
A \ki{nondeterministic finite automaton} (NFA)  \inddef{nondeterministic finite automaton}
\inddef{finite automaton!nondeterministic} is a  system  
${\mathrm A}=(Q,\Sigma, E,I,F)$, where

$\bullet$ $Q$ is a finite, nonempty set of  \ki{states}\inddef{state!of finite automaton},

$\bullet$ $\Sigma$ is the  \ki{input alphabet},\inddef{input alphabet!of finite automaton}

$\bullet$ $E$ is the set of \ki{transitions}\inddef{transition!of finite automaton} (or of
\textit{edges})\index{edge!of finite automaton|see{transition}}, where $E\subseteq Q\times \Sigma \times Q $, 

$\bullet$ $I\subseteq Q$ is the set of \ki{initial states}\inddef{initial state!of finite automaton},

$\bullet$ $F\subseteq Q$ is the set of \ki{final states.}\inddef{final state!of finite automaton} 
\end{defi}

An NFA is in fact a directed, labelled graph, whose vertices are the states and  there is a (directed) edge labelled with $a$ from vertex $p$ to vertex $q$ if  $(p,a,q)\in E$.
Among vertices  some are initial and some  final states.  Initial states are marked by a small arrow entering the corresponding vertex, while the final states are marked with double circles. 
If two vertices are joined by two edges with the same direction then these can be replaced by only one edge labelled with two letters. This graph can be called a transition graph.
 
\begin{pelda}\label{KZa2k1a}
Let ${\mathrm A}=(Q,\Sigma, E,I,F)$, where $Q= \{q_0,q_1,q_2 \}$, $\;\Sigma=\{0,1,2 \}$,
$E=\big\{ (q_0,0,q_0), (q_0,1,q_1), (q_0,2,q_2),$\\   
    \hspace*{1.2cm}  $(q_1,0,q_1), (q_1,1,q_2), (q_1,2,q_0),$\\
     \hspace*{1.2cm} $ (q_2,0,q_2), (q_2,1,q_0), (q_2,2,q_1)\big\}$

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(75.00,45.00)
\emline{7.00}{35.00}{1}{7.00}{45.00}{2}
\emline{17.00}{45.00}{3}{17.00}{35.00}{4}
\emline{27.00}{35.00}{5}{27.00}{45.00}{6}
\emline{37.00}{45.00}{7}{37.00}{35.00}{8}
\emline{47.00}{35.00}{9}{47.00}{45.00}{10}
\emline{57.00}{45.00}{11}{57.00}{35.00}{12}
\put(12.00,40.00){\makebox(0,0)[cc]{$a_1$}}
\put(22.00,40.00){\makebox(0,0)[cc]{$a_2$}}
\put(32.00,40.00){\makebox(0,0)[cc]{$a_3$}}
\put(42.00,40.00){\makebox(0,0)[cc]{\ldots}}
\emline{27.00}{23.00}{13}{67.00}{23.00}{14}
\emline{47.00}{5.00}{15}{67.00}{5.00}{16}
\put(67.00,5.00){\vector(1,0){4.00}}
\put(47.00,17.00){\makebox(0,0)[cc]{control unit}}
\put(73.00,40.00){\makebox(0,0)[lc]{input tape}}
\put(75.00,5.00){\makebox(0,0)[lc]{yes/no}}
\emline{27.00}{23.00}{17}{27.00}{11.00}{18}
\emline{27.00}{11.00}{19}{67.00}{11.00}{20}
\emline{67.00}{11.00}{21}{67.00}{23.00}{22}
\emline{47.00}{23.00}{23}{47.00}{29.00}{24}
\emline{47.00}{29.00}{25}{22.00}{29.00}{26}
\put(22.00,29.00){\vector(0,1){6.00}}
\emline{47.00}{5.00}{27}{47.00}{11.00}{28}
\emline{7.00}{45.00}{29}{57.00}{45.00}{30}
\emline{57.00}{35.00}{31}{7.00}{35.00}{32}
\put(52.00,40.00){\makebox(0,0)[cc]{$a_n$}}
\emline{57.00}{35.00}{33}{59.00}{35.00}{34}
\emline{57.00}{45.00}{35}{59.00}{45.00}{36}
\end{picture}
\caption{Finite automaton.}
\label{KZ2ka0}
\end{figure}





  $I=\{ q_0\}$,   $\; F=\{ q_0\} .$

The automaton can be seen in Fig.  \ref{KZ2k1a}.  
\end{pelda}


\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(87.00,69.00)
\put(15.00,20.00){\circle{10.00}}
\put(76.00,20.00){\circle{10.00}}
\put(46.00,55.00){\circle{10.00}}
\put(15.00,20.00){\makebox(0,0)[cc]{$q_0$}}
\put(46.00,55.00){\makebox(0,0)[cc]{$q_1$}}
\put(76.00,19.00){\makebox(0,0)[cc]{$q_2$}}
\bezier{224}(19.00,23.00)(46.00,32.00)(72.00,23.00)
\bezier{224}(72.50,16.20)(46.00,7.00)(19.50,16.20)
\bezier{160}(18.00,24.00)(38.00,35.00)(43.00,51.00)
\bezier{180}(12.00,24.00)(25.00,48.00)(41.00,55.00)
\bezier{164}(73.00,24.00)(52.00,35.00)(49.00,51.00)
\bezier{184}(51.00,55.00)(71.00,48.00)(79.00,24.00)
\put(80.50,20.00){\oval(8.00,6.00)[r]}
\put(69.00,47.00){\makebox(0,0)[cc]{1}}
\put(58.00,38.00){\makebox(0,0)[cc]{2}}
\put(78.25,26.00){\vector(1,-2){1.00}}
\put(49.90,48.20){\vector(-1,3){1.00}}
\put(87.00,19.00){\makebox(0,0)[cc]{0}}
\put(46.00,59.50){\oval(6.00,11.00)[t]}
\put(11.00,20.00){\oval(10.00,6.00)[l]}
\put(9.00,17.00){\vector(1,0){2.00}}
\put(49.00,61.00){\vector(0,-1){2.00}}
\put(42.10,48.20){\vector(1,3){1.00}}
\put(12.90,25.60){\vector(-1,-2){1.00}}
\put(19.50,16.30){\vector(-3,1){1.00}}
\put(70.80,23.50){\vector(2,-1){1.00}}
\put(4.00,19.00){\makebox(0,0)[cc]{0}}
\put(46.00,69.00){\makebox(0,0)[cc]{0}}
\put(26.00,47.00){\makebox(0,0)[cc]{2}}
\put(34.00,38.00){\makebox(0,0)[cc]{1}}
\put(45.00,30.00){\makebox(0,0)[cc]{2}}
\put(45.00,14.00){\makebox(0,0)[cc]{1}}
\put(81.00,17.00){\vector(-1,0){1.00}}
\put(15.00,10.00){\vector(0,1){5.00}}
\put(15.00,20.00){\circle{7.21}}
\end{picture}
\caption{The finite automaton of Example  \ref{KZa2k1a}.}
\label{KZ2k1a}
\end{figure}



In the case of an edge  $(p,a,q)$ vertex  $p$ is the start-vertex, $q$ the end-vertex and   $a$ the label. 
Now define the notion of the \ki{walk} as in the case of graphs. A sequence 
 \[ (q_0,a_1,q_1),(q_1,a_2,q_2), \ldots, (q_{n-2},a_{n-1},q_{n-1}),(q_{n-1},a_n,q_n) \]
of edges of a NFA is a walk with the label  $a_1a_2\ldots a_n$. 
If  $n=0$ then $q_0=q_n$ and $a_1a_2\ldots a_n=\varepsilon$. Such a walk is called an \ki{empty walk.}  
For a walk the notation
\[ q_0 \overset{a_1}{\longrightarrow} q_1 \overset{a_2}{\longrightarrow} q_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{n-1}}{\longrightarrow} q_{n-1} \overset{a_n}{\longrightarrow} q_n  \]
will be used, or if $w=a_1a_2\ldots a_n$ then we write shortly  $q_0 \overset{w}{\longrightarrow} q_n$. Here $q_0$ is the start-vertex and   $ q_n$ the end-vertex of the walk. The states in a walk are not necessary distinct.
   
A walk is  \ki{productive} if its start-vertex is an initial state and its end-vertex is a final state.   
 We say that an NFA  \ki{accepts} or \ki{recognizes} a word if this word is the label of a productive walk. 
 The empty word $\varepsilon$ is accepted by an NFA if there is an empty productive walk, 
 i.e. there is an initial state which is also a final state.


\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(121.00,39.00)
\put(21.00,34.00){\circle{8.00}}
\put(41.00,34.00){\circle{8.00}}
\put(21.00,34.00){\circle{10.00}}
\put(41.00,34.00){\circle{10.00}}
\put(21.00,14.00){\circle{10.00}}
\put(12.00,34.00){\vector(1,0){4.00}}
\put(12.00,14.00){\vector(1,0){4.00}}
\put(21.00,19.00){\vector(0,1){10.00}}
\put(26.00,34.00){\vector(1,0){10.00}}
\put(44.50,34.50){\oval(13.00,7.00)[r]}
\put(45.00,38.00){\vector(-1,0){1.00}}
\put(21.00,14.00){\makebox(0,0)[cc]{$q_0$}}
\put(21.00,34.00){\makebox(0,0)[cc]{$q_1$}}
\put(41.00,34.00){\makebox(0,0)[cc]{$q_2$}}
\put(23.00,24.00){\makebox(0,0)[lc]{0}}
\put(31.00,36.00){\makebox(0,0)[cc]{1}}
\put(55.00,35.00){\makebox(0,0)[cc]{0,1}}
\put(72.00,14.00){\circle{10.00}}
\put(95.00,14.00){\circle{10.00}}
\put(116.00,14.00){\circle{10.00}}
\put(116.00,14.00){\circle{8.00}}
\put(62.00,14.00){\vector(1,0){5.00}}
\put(77.00,14.00){\vector(1,0){13.00}}
\put(100.00,14.00){\vector(1,0){11.00}}
\put(72.50,18.50){\oval(7.00,15.00)[t]}
\put(116.50,18.50){\oval(7.00,15.00)[t]}
\put(120.00,19.00){\vector(0,-1){2.00}}
\put(76.00,19.00){\vector(0,-1){2.00}}
\put(72.00,14.00){\makebox(0,0)[cc]{$q_0$}}
\put(95.00,14.00){\makebox(0,0)[cc]{$q_1$}}
\put(116.00,14.00){\makebox(0,0)[cc]{$q_2$}}
\put(84.00,16.00){\makebox(0,0)[cc]{0}}
\put(105.00,16.00){\makebox(0,0)[cc]{1}}
\put(72.00,29.00){\makebox(0,0)[cc]{0,1}}
\put(116.00,29.00){\makebox(0,0)[cc]{0,1}}
\put(21.00,4.00){\makebox(0,0)[cc]{${\mathrm A}$}}
\put(95.00,4.00){\makebox(0,0)[cc]{${\mathrm B}$}}
\end{picture}
\caption{Nondeterministic finite automata.}
\label{KZ2k2a}
\end{figure}
\begin{figure}
\begin{minipage}[!h]{7cm}
\begin{center}\begin{tabular}{c|cc}
$\delta$  &  0           &  1  \\ \hline
$q_0$     &  $\{q_1 \}$  &  $\emptyset$ \\
$q_1$     &  $\emptyset$ &  $\{q_2\}$ \\
$q_2$     &  $\{q_2 \}$  &  $\{q_2\}$ \\
\multicolumn{3}{c}{$ $}\\
\multicolumn{3}{c}{{\normalsize {A}}}\\
\end{tabular}\end{center}
\end{minipage}
\begin{minipage}[t]{6cm}
\begin{center}\begin{tabular}{c|cc}
$\delta$  &  0              &  1 \\ \hline
$q_0$     &  $\{q_0,q_1 \}$ &  $\{q_0\}$\\
$q_1$     &  $\emptyset$    &  $\{q_2\}$ \\
$q_2$     &  $\{q_2 \}$     &  $\{q_2\}$\\
\multicolumn{3}{c}{$ $}\\ 
\multicolumn{3}{c}{\normalsize {B}}\\ 
\end{tabular}\end{center}
\end{minipage}
\caption{Transition tables of the NFA  in Fig. \ref{KZ2k2a}.}
\label{KZatmenet}
\end{figure}
   
 The set of words accepted by an NFA will be called the language accepted by this NFA. 
 The  \ki{language accepted} or \ki{recognized}  by  NFA ${\mathrm A}$ is
\[L({\mathrm A})=\Big\{  w\in \Sigma^* \;|\; \exists p\in I, \; \exists q\in F, \; 
\exists p \overset{w}{\longrightarrow} q \Big\} \koz . \] 
The NFA ${\mathrm A}_1$  and ${\mathrm A}_2$ are \ki{equivalent} if $L({\mathrm A}_1)=L({\mathrm A}_2)$.

Sometimes it is useful the following \ki{transition function:} 
\[ \delta: Q\times\Sigma \rightarrow  P(Q), \quad  \delta(p,a)= \{ q \in Q \; | \; (p,a,q)\in E\} \koz . \]

This function associate to a state $p$ and input letter $a$ the set of states in which the automaton can go if its current state is $p$ and the head is on input letter $a$. 

Denote by  $|H|$ the cardinal (the number of elements) of  $H$.\footnote{ The same notation is used 
for the cardinal of a set and length of a word, but this is no matter of confusion because 
for word we use lowercase letters and for set capital letters. The only exception is  $\delta (q,a)$, 
but this could not be confused with a word.} An NFA is a  
\ki{deterministic finite automaton} 
(DFA)\inddef{deterministic finite automaton}\inddef{finite automaton!deterministic} if  
\[ |I|=1\; \textrm{ and }\; |\delta(q,a)|\le 1, \; \forall q\in Q, \; \forall a\in \Sigma \koz .\]
 In Fig. \ref{KZ2k1a} a DFA can be seen.

Condition  $|\delta(q,a)|\le 1$ can be replaced by
\[ (p,a,q) \in E, \; (p,a,r)\in E \; \Longrightarrow \; q=r \koz , \forall p,q,r\in Q, \forall a\in \Sigma\koz.\]
If for a DFA $|\delta(q,a)|= 1$ for each  state $q\in Q$ and for each letter $a\in \Sigma$ then it is called a  \ki{complete DFA.} \inddef{finite automaton!complete deterministic}

Every DFA can be transformed in a complete DFA by introducing a new state, which can be called a snare state. Let 
${\mathrm A}=(Q,\Sigma, E,\{ q_0\},F)$ be a DFA.  An equivalent and complete DFA will be ${\mathrm A'}=(Q\cup\{s\},\Sigma, E',\{ q_0\},F)$, where $s$
 is the new state and  $E'=E\cup \big\{ (p,a,s)\;|\; \delta(p,a)=\emptyset, p\in Q, a\in \Sigma\big\}\cup\big\{(s,a,s)\;|\; a\in \Sigma\big\}$. It is easy to see that  $L({\mathrm A})=L({\mathrm A'})$.
  
Using the transition function we can easily define the transition table.  The rows of this table are indexed by the elements of $Q$, its columns by the elements of  $\Sigma$. At the intersection of row  $q\in Q$ and column $a\in \Sigma$ we put $\delta(q,a)$.  
In the case of Fig. \ref{KZ2k1a}, the transition table is:

\begin{center}
\begin{tabular}{c|ccc}
$\delta$& 0     & 1     & 2 \\ \hline
$q_0$   & $\{q_0\}$ & $\{q_1\}$ & $\{q_2\}$ \\
$q_1$   & $\{q_1\}$ & $\{q_2\}$ & $\{q_0\}$ \\
$q_2$   & $\{q_2\}$ & $\{q_0\}$ & $\{q_1\}$ \\
\end{tabular}
\end{center}

The NFA in Fig. \ref{KZ2k2a} are not deterministic: the first (automaton  A) 
has two initial states, the second (automaton  B) has two transitions with $0$ 
from state $q_0$ (to states  $q_0$  and $q_1$). The transition 
table of these two automata are in Fig. \ref{KZatmenet}.
$L({\mathrm A})$  is set of words over $\Sigma =\{0,1\}$ which do not 
begin with two zeroes  (of course  $\varepsilon$ is in language), 
$L({\mathrm B})$ is the set of words which contain $01$ as a subword.

\subsubsection{Eliminating inaccessible states.}\index{state!inaccessible}

Let ${\mathrm A}= (Q, \Sigma, E,I,F)$ be a finite automaton. A state is 
\emph{accessible} if it is on a walk which starts by an initial state.
The following algorithm determines the inaccessible states building  a sequence $U_0$, $U_1$, $U_2, 
\dots$ of sets, where $U_0$ is the set of initial states, and for any $i\ge 1$  
$U_i$ is the set of accessible states, which are at distance at most  $i$ from an initial state.
 
\begin{algN}{\textsc{Inaccessible-States}(A)\index{inacces@\textsc{Inaccessible-States}}}\\
1 \' $U_0\leftarrow I$\\
2 \'  $i\leftarrow 0$\\
3 \' \key{repeat}\\  
4 \'  \quad \= $i\leftarrow i+1$\\
5 \'        \>  \key{for} \= all  $q \in U_{i-1}$  \\ 
6 \'        \>            \>  \key{do} \key{for} \= all $ a \in \Sigma$ \\   
7 \'        \>            \>                     \> \key{do}   $U_i\leftarrow U_{i-1}\cup \delta(q,a)$\\
8 \'\key{until}  $U_i=U_{i-1}$  \\
9 \'  $U\leftarrow Q\setminus U_{i}$\\
10 \' \key{return} $U$ 
\end{algN}

The inaccessible states of the automaton can be eliminated without changing the accepted language.


If $|Q|=n$ and $|\Sigma|=m$ then the running time of the algorithm (the number of steps) 
in the worst case is $O(n^2m)$, because the number of steps in the two embedded loops is at most $nm$ and in the loop  \key{repeat} at most  $n$. 

Set $U$ has the property that $L({\mathrm A})\not=\emptyset$ if and only if $U\cap F\not= \emptyset$. 
The above algorithm can be extended by inserting the $U\cap F\not= \emptyset$ 
condition to decide if language $L({\mathrm A})$ is or not empty.


\subsubsection{Eliminating nonproductive states.}\index{state!nonproductive}
Let  ${\mathrm A}= (Q, \Sigma, E,I,F)$ be a finite automaton.  
A state is  \emph{productive} if it is on a walk which ends in a terminal state. 
For finding the productive states the following algorithm uses the function $\delta^{-1}$: 

\smallskip
\centerline{$ \delta^{-1}: Q\times \Sigma \rightarrow {\mathcal  P}(Q), 
\quad \delta^{-1}(p,a)=\{q \;|\; (q,a,p)\in E  \} \koz . $}

\smallskip
\noindent This function for a state  $p$ and a letter $a$ gives the set of all states 
from which using this letter $a$ the automaton  can go into the state $p$.
 
\begin{algN}{\textsc{Nonproductive-States}(A)\index{nonproduct@\textsc{Nonproductive-States}}}\\
1 \'  $V_0\leftarrow F$\\
2 \'  $i\leftarrow 0$\\
3 \'  \key{repeat}\\  
4 \'     \quad \= $i\leftarrow i+1$\\
5 \'             \>  \key{for} \=  all  $p \in V_{i-1}$\\ 
6 \'             \>            \> \key{do} \key{for} \= all $ a \in \Sigma$\\   
7 \'             \>            \>                    \> \key{do} $V_i\leftarrow V_{i-1}\cup \delta^{-1}(p,a)$

\algnewpage
8 \'  \key{until} $V_i=V_{i-1}$  \\
9 \'   $V\leftarrow Q\setminus V_{i}$\\
10 \' \key{return} $V$
\end{algN}
The nonproductive states of the automaton can be eliminated without changing the accepted language.

If $n$ is the number of states, $m$ the number of letters in the alphabet, 
then the running time of the algorithm is also $O(n^2m)$ as in the case 
of the algorithm \textsc{Inaccessible-States}.

The set $V$ given by the algorithm has the property that $L({\mathrm A})\not=\emptyset$ 
if and only if  $V\cap I\not= \emptyset$. So, by a little modification 
it can be used to decide if language $L({\mathrm A})$ is or not empty.

\subsection{Transforming nondeterministic finite automata}

As follows we will show that any NFA can be transformed in an equivalent DFA.

\begin{tetel}\label{KZnemdetdet}
For any NFA one may construct an equivalent DFA.
\end{tetel}
\begin{biz}
Let ${\mathrm A}=(Q,\Sigma, E,I, F)$ be an NFA. Define a DFA  
 $\overline{{\mathrm A}}=(\overline{Q}, \Sigma, \overline{E}, \overline{I}, \overline{F})$, where
 
\smallskip
$\bullet$ $\overline{Q}={\mathcal P}(Q)\setminus \emptyset$,

$\bullet$ edges of $ \overline{E}$ are those triplets $(S,a,R)$ for which  
$R,S\in \overline{Q}$ are not empty, $a\in \Sigma$ and $R=\displaystyle\bigcup_{p\in S} \delta(p,a)$,

$\bullet$ $\overline{I}=\{ I \},$
			 
$\bullet$ $\overline{F}= \{ S\subseteq Q \;|\; S\cap F\not= \emptyset\}.$

We prove that $L({\mathrm A})=L(\overline{{\mathrm A}})$. 

$a)$ First prove that $L({\mathrm A})\subseteq L(\overline{{\mathrm A}})$.
Let $w=a_1a_2\ldots a_k\in L({\mathrm A})$. Then there exists a walk  
\[ q_0 \overset{a_1}{\longrightarrow} q_1 \overset{a_2}{\longrightarrow} q_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{k-1}}{\longrightarrow} q_{k-1} \overset{a_k}{\longrightarrow} q_k, \;\; q_0\in I, \;q_k\in F \koz .\]
Using the transition function $\overline{\delta}$ of  NFA $\overline{{\mathrm A}}$ we construct the sets
  $S_0=\{q_0\}$, $\overline{\delta}(S_0,a_1)=S_1$, \ldots 
$\overline{\delta}(S_{k-1}, a_k)=S_k$. Then $q_1\in S_1, \ldots ,
q_k\in S_k$  and since  $q_k\in F$ we get $S_k\cap F\not= \emptyset$,
so $S_k \in \overline{F}$. Thus, there exists a walk 
\[S_0 \overset{a_1}{\longrightarrow} S_1 \overset{a_2}{\longrightarrow} S_2 \overset{a_3}{\longrightarrow} \;\cdots\; 
  \overset{a_{k-1}}{\longrightarrow}  S_{k-1} \overset{a_k}{\longrightarrow} S_k, \;\; S_0\subseteq I, \;S_k\in \overline{F} \koz .\]
There are sets  $S'_0, \ldots, S'_k$ for which $S'_0=I$, and for  $i=0,1,\ldots, k$ we have $S_i\subseteq S'_i$, and  
\[S'_0 \overset{a_1}{\longrightarrow} S'_1 \overset{a_2}{\longrightarrow} S'_2 \overset{a_3}{\longrightarrow} \;\cdots\; 
  \overset{a_{k-1}}{\longrightarrow}  S'_{k-1} \overset{a_k}{\longrightarrow} S'_k \] is a productive walk. Therefore
$w\in L(\overline{{\mathrm A}})$. That is $L({\mathrm A})\subseteq L(\overline{{\mathrm A}})$.

$b)$ Now we show that $L(\overline{{\mathrm A}})\subseteq L({\mathrm A})$.
Let $w=a_1a_2\ldots a_k\in L(\overline{{\mathrm A}})$. Then there is a walk \[ \overline{q_0} \overset{a_1}{\longrightarrow} \overline{q_1} \overset{a_2}{\longrightarrow} \overline{q_2} 
   \overset{a_3}{\longrightarrow} \;\cdots \; \overset{a_{k-1}}{\longrightarrow} \overline{q}_{k-1} 
   \overset{a_k}{\longrightarrow} \overline{q}_k, \;\; 
   \overline{q}_0\in \overline{I}, \;\overline{q}_k\in \overline{F} \koz .\] 
Using the definition of $\overline{F}$ we have $\overline{q}_k \cap F\not= \emptyset$, i.e. there exists $q_k\in \overline{q}_k\cap F$, that is by the definitions of $q_k\in F$ and $\overline{q}_k$ there is  $q_{k-1}$ such that $(q_{k-1},a_k,q_k)\in E$. Similarly, there are the states $q_{k-2}, \ldots, q_1, q_0$ such that $(q_{k-2},a_k,q_{k-1})\in E, \ldots, (q_0,a_1,q_1)\in E,$ where $q_0\in \overline{q}_0=I$, thus, there is a walk   

\centerline{$ q_0 \overset{a_1}{\longrightarrow} q_1 \overset{a_2}{\longrightarrow} q_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{k-1}}{\longrightarrow} q_{k-1} \overset{a_k}{\longrightarrow} q_k, \;\; q_0\in I, \;q_k\in F \koz ,$}

\noindent so  $L(\overline{{\mathrm A}})\subseteq L({\mathrm A})$.
\end{biz}

In constructing DFA we can use the corresponding transition function  $\overline{\delta}$:
\[ \overline{\delta} (\overline{q},a)=\left\{ \bigcup_{q\in \overline{q}}\delta(q,a) \right\},  
\; \forall \overline{q}\in \overline{Q}, \forall a\in \Sigma \koz .\] 
The empty set was excluded from the states, so we used here  $\emptyset$ instead of  $\{\emptyset\}$.

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(74.00,30.00)
\put(16.00,10.00){\circle{10.00}}
\put(39.00,10.00){\circle{10.00}}
\put(60.00,10.00){\circle{10.00}}
\put(60.00,10.00){\circle{8.00}}
\put(6.00,10.00){\vector(1,0){5.00}}
\put(21.00,10.00){\vector(1,0){13.00}}
\put(44.00,10.00){\vector(1,0){11.00}}
\put(16.00,10.00){\makebox(0,0)[cc]{$S_0$}}
\put(39.00,10.00){\makebox(0,0)[cc]{$S_2$}}
\put(60.00,10.00){\makebox(0,0)[cc]{$S_3$}}
\put(28.00,12.00){\makebox(0,0)[cc]{0}}
\put(49.00,12.00){\makebox(0,0)[cc]{1}}
\put(16.00,10.00){\circle{8.00}}
\put(39.00,10.00){\circle{8.00}}
\bezier{212}(16.00,15.00)(39.00,30.00)(59.30,15.60)
\put(58.80,16.00){\vector(1,-1){1.00}}
\put(64.50,9.50){\oval(15.00,7.00)[r]}
\put(65.00,6.00){\vector(-1,0){2.00}}
\put(74.00,10.00){\makebox(0,0)[lc]{$0,1$}}
\put(39.00,25.00){\makebox(0,0)[cc]{1}}
\end{picture}
\caption{The equivalent DFA with NFA  ${\mathrm A}$ in Fig.  \ref{KZ2k2a}.}
\label{KZ2k3a}
\end{figure}


\begin{pelda}
Apply  Theorem  \ref{KZnemdetdet} to transform  NFA ${\mathrm A}$ in Fig. \ref{KZ2k2a}.  
Introduce the following notation for the states of the DFA: 

\medskip
$\begin{array}{llll}
S_0:=\{q_0,q_1\}, \qquad &  S_1:=\{q_0\},     \qquad &   S_2:=\{q_1\},    \qquad  & S_3:=\{q_2\},  \\
S_4:=\{q_0,q_2\}, \qquad &   S_5:=\{q_1,q_2\}, \qquad & S_6:=\{q_0,q_1,q_2\}\koz,     &  
\end{array}$ 

\medskip
\noindent where $S_0$ is the initial state. Using the transition function we get the transition table:

\begin{center}
\begin{tabular}{c|cc}
$\overline{\delta}$& $\;\; 0\;\;$ & $\;\;1\;\; $  \\ \hline
$S_0$  &  $\{S_2\}$    &  $\{S_3\}$ \\
$S_1$  &  $\{S_2\}$    &  $\emptyset$ \\
$S_2$  &  $\emptyset$  &  $\{S_3\}$ \\
$S_3$  &  $\{S_3\}$    &  $\{S_3\}$ \\
$S_4$  &  $\{S_5\}$    &  $\{S_3\}$ \\
$S_5$  &  $\{S_3\}$    &  $\{S_3\}$ \\
$S_6$  &  $\{S_5\}$    &  $\{S_3\}$\\
\end{tabular}
\end{center}

\noindent This automaton contains many inaccessible states.  
By algorithm \textsc{Inaccessible-States} we determine the accessible states of DFA:

\smallskip
$U_0=\{S_0\},$ \quad  $ U_1=\{S_0, S_2,S_3\},$  \quad $  U_2=\{S_0, S_2,S_3\}=U_1=U.$

\smallskip
Initial state $S_0$ is also a final state. States  $S_2$  and $S_3$ are final states. 
States $S_1,S_4,S_5,S_6$  are inaccessible and can be removed from the DFA. 
The transition table of the resulted DFA is

\begin{center}
\begin{tabular}{c|cc}
$\overline{\delta}$ &   0    & 1 \\ \hline
$S_0$               & $\{S_2\}$  & $\{S_3\}$ \\
$S_2$               &  $\emptyset$   & $\{S_3\}$ \\ 
$S_3$               &  $\{S_3\}$ &  $\{S_3\}$ 
\end{tabular}
\end{center}

\noindent The corresponding transition graph is in Fig. \ref{KZ2k3a}. 
\end{pelda}

The algorithm given in Theorem \ref{KZnemdetdet} can be simplified. 
It is not necessary to consider all subset of the set of states of NFA. 
The states of DFA $\overline{{\mathrm A}}$ can be obtained successively. Begin with the state 
 $\overline{q}_0=I$ and determine the states $\overline{\delta}(\overline{q}_0,a)$ 
 for all $a\in \Sigma$. For the newly obtained states we determine the states accessible from them. 
 This can be continued until no new states arise.

In our previous example  $\;\overline{q}_0:=\{q_0,q_1\}$ is the initial state.  From this we get
 
\smallskip
$\begin{array}{ll}
\overline{\delta}(\overline{q}_0,0)=\{\overline{q_1}\},\; \textrm{ where } \;\overline{q}_1:=\{q_1\}, &\quad
\overline{\delta}(\overline{q}_0,1)=\{\overline{q}_2\},\; \textrm{ where }  \;\overline{q}_2:=\{q_2\},\\
\overline{\delta}(\overline{q}_1,0)=\emptyset, &\quad \overline{\delta}(\overline{q}_1,1)=\{\overline{q}_2\},\\
\overline{\delta}(\overline{q}_2,0)=\{\overline{q}_2\}, &\quad  \overline{\delta}(\overline{q}_2,1)=\{\overline{q}_2\}.
\end{array}$

\smallskip\noindent The transition table is 
\begin{center}
\begin{tabular}{c|cc}
$\overline{\delta}$ &   0    & 1 \\ \hline
$\overline{q}_0$ & $\{\overline{q}_1\}$  & $\{\overline{q}_2\}$ \\
$\overline{q}_1$ &  $\emptyset$   & $\{\overline{q}_2\}$ \\ 
$\overline{q}_2$  &  $\{\overline{q}_2\}$ &  $\{\overline{q}_2\}$ 
\end{tabular}
\end{center} 
 
 \noindent which is the same  (excepted the notation) as before.
 
The next algorithm will construct for an NFA  ${\mathrm A}= (Q, \Sigma, E, I,F)$ 
the transition table $M$ of the equivalent DFA  $\overline{{\mathrm A}}= 
(\overline{Q}, \Sigma, \overline{E}, \overline{I},\overline{F})$, 
but without to determine the final states (which can easily be included). 
  Value of {\sc IsIn}($\overline{q},\overline{Q}$) in the algorithm is true if state  
   $\overline{q}$ is already in $\overline{Q}$ and is false otherwise. 
   Let $a_1,a_2, \ldots, a_m$ be an ordered list of the letters of $\Sigma$.

 
\begin{algN}{NFA-DFA(A)}\inddef{NFA-DFA}
1 \' $\overline{q}_0\leftarrow I$ \\
2 \' $\overline{Q}\leftarrow \{\overline{q}_0\}$\\
3 \' $i\leftarrow 0$          \`   $\rhd$       $i$ counts the rows.\\
4 \' $k\leftarrow 0$          \` $\rhd$       $k$ counts the states.

\algnewpageN
5 \' \key{repeat} \=\\  
6 \'              \>  \key{for} \= $j=1, 2, \ldots, m$             \` $\rhd$         $j$ counts the columns.\\
7  \'             \>            \> \key{do} \= $\overline{q} \leftarrow {\displaystyle\bigcup_{p\in\overline{q}_i}\delta (p,a_j)}$\\ 
8 \'              \>            \> \>\key{if} \= $ \overline{q} \not= \emptyset $\\ 
9 \'              \>           \> \> \>\key{then} \= \key{if} \= {\sc IsIn}($\overline{q},\overline{Q}$)\\
10 \'             \>           \> \> \>            \>         \> \key{then} \= $M[i,j]\leftarrow \{\overline{q}\}$\\
11 \'             \>           \> \> \>            \>         \> \key{else} \> $k\leftarrow k+1$ \\ 
12 \'             \>           \> \>  \>           \>        \>             \> $\overline{q}_k\leftarrow \overline{q}$\\ 
13 \'             \>          \> \> \>             \>       \>             \>  $M[i,j]\leftarrow \{\overline{q}_k\} $\\
14 \'             \>  \>    \>  \>        \>    \> \>   $\overline{Q} \leftarrow \overline{Q}\cup \{\overline{q}_k\}  $\\
15 \'             \>  \>  \> \> \key{else} \> $M[i,j]\leftarrow \emptyset$ \\
16 \'             \> $ i\leftarrow i+1$\\
17 \' \key{until}  $i=k+1$  \\
18 \' \key{return} transition table  $M$ of $\overline{{\mathrm A}}$ 
\end{algN}

 
Since loop \key{repeat} is executed  as many times as the number of states of new automaton, 
in worst case the running time can be exponential, because, if the number of states in NFA is $n$, 
then DFA can have even $2^n-1$ states. (The number of subsets of a set of  
$n$ elements is $2^n$, including the empty set.)

Theorem  \ref{KZnemdetdet} will have it that to any NFA one may construct an equivalent DFA. 
Conversely, any DFA is also an NFA by definition. So, the nondeterministic 
finite automata accepts the same class of languages as the deterministic finite automata.

\subsection{Equivalence of deterministic finite automata} \index{automata!equivalence}
In this subsection we will use complete deterministic finite automata only.  In this case 
 $\delta(q,a)$  has a single element. In formulae, sometimes, instead of set  $\delta(q,a)$ 
 we will use its single element. We introduce for a set $A=\{a\}$ the function  
 $elem(A)$  which give us the single element of set  $A$, so $elem(A)=a$. \label{egyelemu}
Using walks which begin with the initial state and have the same label in two DFA's 
we can determine the equivalence of these DFA's. If only one  of these walks ends in a final state, 
then they could  not be equivalent. 

 Consider two DFA's over the same alphabet 
$A=(Q,\Sigma, E,\{q_0\}, F)$  and  $A'=(Q', \Sigma, E',\{q_0'\},F')$. 
We are interested to determine if they are or not  equivalent. 
We construct a table with elements of form  $(q,q')$, where $q\in Q$ and $q'\in Q'$.
Beginning with the second column of the table, we associate a column to each letter of the alphabet $\Sigma$.
If the first element of the $i$th row is  $(q,q')$ then at the cross of $i$th  
row and the column associated to letter $a$  will be the pair 
$\Big(elem\big(\delta(q,a)\big), elem\big(\delta'(q',a)\big)\Big)$. 
\begin{center}
\begin{tabular}{|c|ccc|c}
         \multicolumn{2}{l}{ }  &   \ldots \quad $a$  \quad\ldots &   \multicolumn{2}{r}{}\\ \hline
  \ldots   &        &       $\ldots $             &   \\ 
  $(q,q')$  &        & $\Big(elem\big(\delta(q,a)\big), elem\big(\delta'(q',a)\big)\Big)$&     \\
  \ldots   &        &       $\ldots $             &\\ \hline
\end{tabular}
\end{center}
In the first column of the  first row we put $(q_0, q_0')$ and complete the first row using the above method.  
If in the first row in any column there occur a pair of states from which one is a final state and the other not then 
the algorithm ends, the two automata are \ki{not equivalent.}  
If there is no such a pair of states, every new pair is written in the first column. 
The algorithm continues with the next unfilled row. 
If no new pair of states occurs in the table and for each pair both of states are final or both are not, 
then the algorithm ends and the two DFA are \ki{equivalent.}  

If $|Q|=n$,  $\;|Q'|=n'$  and $|\Sigma|=m$ then taking into account 
that in worst case loop \key{repeat} is executed  $nn'$ times, 
loop \key{for} $m$ times, the running time of the algorithm in worst case will be $O(nn'm)$, 
or if $n=n'$ then $O(n^2m)$. 

Our algorithm was described to determine the equivalence of two complete DFA's. 
If we have to determine the equivalence of two NFA's, first we transform them into 
complete DFA's and after this we can apply the above algorithm.

\begin{algN}{DFA-Equivalence(${\mathrm A}, {\mathrm A}'$)}
\inddef{DFA-Equivalence@\textsc{DFA-Equivalence}}
1 \' write in the first column of the first row the pair $(q_0,q'_0)$ \\
2 \' $i\leftarrow 0$ \\
3 \' \key{repeat} \= \\
4 \'              \> $i\leftarrow i+1$\\
5 \'              \>  let $(q,q')$ be the pair in the first column of the $i$th row\\
6 \'              \> \key{for} \= all  $a\in \Sigma$ \\
7 \'              \> \> \key{do} \= write in the column associated to $a$ in the $i$th row \\
  \'              \> \>          \> the pair $\Big(elem\big(\delta(q,a)\big), elem\big(\delta'(q',a)\big)\Big)$  \\ 	 
8 \'              \> \> \> \key{if} \= one state in $\Big(elem\big(\delta(q,a)\big), elem\big(\delta'(q',a)\big)\Big)$ is final \\
                  \> \> \>          \> and the other not \\ 
9 \'              \> \>  \>       \> \key{then} \= \key{return} \textsc{no} \\  
10 \'             \> \>  \>       \> \key{else} \> write pair $\Big(elem\big(\delta(q,a)\big), elem\big(\delta'(q',a)\big)\Big)$ \\ 
                  \> \>  \>       \>            \> in the next empty row of the first column,\\
   \'	          \> \>  \>       \> \>  if not occurred already in the first column\\
11 \'  \key{until} the first element of $(i+1)$th row becomes empty \\ 
12 \'  \key{return} \textsc{yes}   \\ 
\end{algN}

\begin{figure}[b!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(110.00,40.00)
\put(10.00,10.00){\circle{10.00}}
\put(10.00,35.00){\circle{10.00}}
\put(40.00,10.00){\circle{10.00}}
\put(68.00,10.00){\circle{10.00}}
\put(68.00,35.00){\circle{10.00}}
\put(98.00,10.00){\circle{10.00}}
\put(98.00,35.00){\circle{10.00}}
\put(1.00,35.00){\vector(1,0){4.00}}
\put(58.00,35.00){\vector(1,0){5.00}}
\put(68.00,35.00){\circle{8.00}}
\put(10.00,35.00){\circle{8.00}}
\put(15.00,34.00){\vector(4,-3){25.00}}
\bezier{76}(8.00,30.00)(3.00,23.00)(8.00,15.00)
\bezier{80}(12.00,30.00)(17.00,22.00)(12.00,15.00)
\bezier{100}(14.00,13.00)(25.00,19.00)(36.00,13.00)
\bezier{100}(14.00,7.00)(25.00,1.00)(36.00,7.00)
\put(44.00,9.50){\oval(12.00,7.00)[r]}
\put(45.00,13.00){\vector(-1,0){1.00}}
\put(7.50,29.20){\vector(1,1){1.00}}
\put(12.50,15.40){\vector(-1,-1){1.00}}
\put(35.00,13.60){\vector(2,-1){1.00}}
\put(15.00,6.60){\vector(-2,1){1.00}}
\put(10.00,35.00){\makebox(0,0)[cc]{$q_0$}}
\put(10.00,10.00){\makebox(0,0)[cc]{$q_1$}}
\put(40.00,10.00){\makebox(0,0)[cc]{$q_2$}}
\put(29.00,27.00){\makebox(0,0)[cc]{$a$}}
\put(8.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(16.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(25.00,18.00){\makebox(0,0)[cc]{$a$}}
\put(25.00,7.00){\makebox(0,0)[cc]{$a$}}
\put(52.00,9.00){\makebox(0,0)[cc]{$b$}}
\put(93.00,35.00){\vector(-1,0){20.00}}
\put(73.00,34.00){\vector(1,-1){21.00}}
\put(73.00,10.00){\vector(1,0){20.00}}
\bezier{84}(65.00,14.00)(59.00,21.00)(65.00,31.00)
\bezier{84}(72.00,14.00)(78.00,22.00)(71.70,31.30)
\bezier{76}(95.00,14.00)(91.00,22.00)(95.00,31.00)
\bezier{80}(102.00,14.00)(107.00,22.00)(101.70,31.30)
\put(102.00,9.50){\oval(12.00,7.00)[r]}
\put(102.50,6.00){\vector(-1,0){1.00}}
\put(94.00,28.00){\vector(1,3){1.00}}
\put(102.60,14.60){\vector(-1,-1){1.00}}
\put(72.60,14.60){\vector(-1,-1){1.00}}
\put(64.20,29.30){\vector(1,2){1.00}}
\put(68.00,35.00){\makebox(0,0)[cc]{$p_0$}}
\put(98.00,35.00){\makebox(0,0)[cc]{$p_2$}}
\put(68.00,10.00){\makebox(0,0)[cc]{$p_1$}}
\put(98.00,10.00){\makebox(0,0)[cc]{$p_3$}}
\put(83.00,37.00){\makebox(0,0)[cc]{$b$}}
\put(83.00,26.00){\makebox(0,0)[cc]{$a$}}
\put(83.00,12.00){\makebox(0,0)[cc]{$a$}}
\put(64.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(77.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(95.00,22.00){\makebox(0,0)[cc]{$a$}}
\put(106.00,22.00){\makebox(0,0)[cc]{$a$}}
\put(110.00,9.00){\makebox(0,0)[cc]{$b$}}
\end{picture}
\caption{Equivalent DFA's (Example \ref{KZa2ka4}).} \label{KZ2k4a} 
\end{figure}

\begin{pelda}\label{KZa2ka4}
Determine if the two DFA's in Fig. \ref{KZ2k4a} are equivalent or not. The algorithm gives the table
\begin{center}
\begin{tabular}{|c|ccc|c}
   \multicolumn{2}{c}{\qquad\qquad$a$}  & $b$  &\multicolumn{2}{l}{}\\ \hline  
$(q_0,p_0)$  & $(q_2, p_3)$ & $(q_1,p_1)$ &\\ 
$(q_2,p_3)$  & $(q_1, p_2)$ & $(q_2,p_3)$ &\\ 
$(q_1,p_1)$  & $(q_2, p_3)$ & $(q_0,p_0)$ &\\ 
$(q_1,p_2)$  & $(q_2, p_3)$ & $(q_0,p_0)$ &\\ \hline 
\end{tabular}
\end{center}

\begin{figure}[hb!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(110.00,40.00)
\put(10.00,10.00){\circle{10.00}}
\put(10.00,35.00){\circle{10.00}}
\put(40.00,10.00){\circle{10.00}}
\put(68.00,10.00){\circle{10.00}}
\put(68.00,35.00){\circle{10.00}}
\put(98.00,10.00){\circle{10.00}}
\put(98.00,35.00){\circle{10.00}}
\put(1.00,35.00){\vector(1,0){4.00}}
\put(58.00,35.00){\vector(1,0){5.00}}
\put(68.00,35.00){\circle{8.00}}
\put(10.00,35.00){\circle{8.00}}
\put(15.00,34.00){\vector(4,-3){25.00}}
\bezier{76}(8.00,30.00)(3.00,23.00)(8.00,15.00)
\bezier{80}(12.00,30.00)(17.00,22.00)(12.00,15.00)
\bezier{100}(14.00,13.00)(25.00,19.00)(36.00,13.00)
\bezier{100}(14.00,7.00)(25.00,1.00)(36.00,7.00)
\put(44.00,9.50){\oval(12.00,7.00)[r]}
\put(45.00,13.00){\vector(-1,0){1.00}}
\put(7.50,29.20){\vector(1,1){1.00}}
\put(12.50,15.40){\vector(-1,-1){1.00}}
\put(35.00,13.60){\vector(2,-1){1.00}}
\put(15.00,6.60){\vector(-2,1){1.00}}
\put(10.00,35.00){\makebox(0,0)[cc]{$q_0$}}
\put(10.00,10.00){\makebox(0,0)[cc]{$q_1$}}
\put(40.00,10.00){\makebox(0,0)[cc]{$q_2$}}
\put(29.00,27.00){\makebox(0,0)[cc]{$b$}}
\put(8.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(16.00,22.00){\makebox(0,0)[cc]{$a$}}
\put(25.00,18.00){\makebox(0,0)[cc]{$a$}}
\put(25.00,7.00){\makebox(0,0)[cc]{$a$}}
\put(52.00,9.00){\makebox(0,0)[cc]{$b$}}
\put(93.00,35.00){\vector(-1,0){20.00}}
\put(73.00,34.00){\vector(1,-1){21.00}}
\put(73.00,10.00){\vector(1,0){20.00}}
\bezier{84}(65.00,14.00)(59.00,21.00)(65.00,31.00)
\bezier{84}(72.00,14.00)(78.00,22.00)(71.70,31.30)
\bezier{76}(95.00,14.00)(91.00,22.00)(95.00,31.00)
\bezier{80}(102.00,14.00)(107.00,22.00)(101.70,31.30)
\put(102.00,9.50){\oval(12.00,7.00)[r]}
\put(102.50,6.00){\vector(-1,0){1.00}}
\put(94.00,28.00){\vector(1,3){1.00}}
\put(102.60,14.60){\vector(-1,-1){1.00}}
\put(72.60,14.60){\vector(-1,-1){1.00}}
\put(64.20,29.30){\vector(1,2){1.00}}
\put(68.00,35.00){\makebox(0,0)[cc]{$p_0$}}
\put(98.00,35.00){\makebox(0,0)[cc]{$p_2$}}
\put(68.00,10.00){\makebox(0,0)[cc]{$p_1$}}
\put(98.00,10.00){\makebox(0,0)[cc]{$p_3$}}
\put(83.00,37.00){\makebox(0,0)[cc]{$b$}}
\put(83.00,26.00){\makebox(0,0)[cc]{$a$}}
\put(83.00,12.00){\makebox(0,0)[cc]{$a$}}
\put(64.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(77.00,22.00){\makebox(0,0)[cc]{$b$}}
\put(95.00,22.00){\makebox(0,0)[cc]{$a$}}
\put(106.00,22.00){\makebox(0,0)[cc]{$a$}}
\put(110.00,9.00){\makebox(0,0)[cc]{$b$}}
\end{picture}
\caption{Non equivalent DFA's (Example \ref{KZa2k4a1}).}
 \label{KZ2k4a1}
\end{figure}
The two DFA's are equivalent because all possible pairs of states are considered and in every pair both states are final 
or both are not final.
\end{pelda}
\begin{pelda}\label{KZa2k4a1}
The table of the two DFA's in Fig. \ref{KZ2k4a1} is: \\
\begin{center}
\begin{tabular}{|c|ccc|c}
             \multicolumn{2}{c}{\qquad\qquad $a$}   &  $b$  & \multicolumn{2}{r}{}  \\ \hline  
$(q_0,p_0)$  & $(q_1, p_3)$ & $(q_2,p_1)$ &\\
$(q_1,p_3)$  & $(q_2, p_2)$ & $(q_0,p_3)$ &\\
$(q_2,p_1)$  &  & & \\ 
$(q_2,p_2)$  &  & & \\ \hline
\end{tabular}
\end{center}
These two DFA's are not equivalent, because in the last column of the second row in the pair $(q_0,p_3)$ the first state is final and the second not. 
\end{pelda}

\subsection{Equivalence of finite automata and regular languages.}

We have seen that NFA's accept the same class of languages as DFA's.  The following theorem states that this class is that of regular languages. 

\begin{tetel} \label{KZdetreg1}
   If $L$ is a language accepted by a DFA, then one may construct a regular grammar which generates language  $L$.
\end{tetel}
\begin{biz}
    Let ${\mathrm A}=(Q,\Sigma,E, \{q_0\}, F)$ be the DFA accepting language $L$, that is $L=L({\mathrm A})$. Define the regular grammar $G=(Q, \Sigma, P, q_0)$  with the productions:

$\bullet$ If $(p,a,q)\in E$ for $p,q\in Q$ and  $a\in \Sigma$, then put production $p\rightarrow aq$ in  $P$.

$\bullet$ If $(p,a,q)\in E$  and $q\in F$, then put also production $p\rightarrow a$ in $P$.

Prove that $L(G)=L({\mathrm A})\setminus \{\varepsilon\}$. 

Let $u=a_1a_2\ldots a_n\in L({\mathrm A})$ and  $u\not= \varepsilon$. Thus, since  
${\mathrm A}$ accepts word $u$, there is a walk
\[ q_0 \overset{a_1}{\longrightarrow} q_1 \overset{a_2}{\longrightarrow} q_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{n-1}}{\longrightarrow} q_{n-1} \overset{a_n}{\longrightarrow} q_n, \; \;q_n\in F \koz .\]
Then there are in  $P$ the productions
\[q_0\rightarrow a_1q_1,\;\; q_1\rightarrow a_2q_2, \;\;\ldots,\;\;q_{n-2}\rightarrow a_{n-1}q_{n-1},\;\; q_{n-1}\rightarrow a_n\] 
(in the right-hand side of the last production $q_n$ does not occur, because $q_n\in F$), so there is the derivation 
\[q_0\Longrightarrow a_1q_1\Longrightarrow a_1a_2q_2\Longrightarrow \ldots 
\Longrightarrow a_1a_2\ldots a_{n-1}q_{n-1}\Longrightarrow a_1a_2\ldots a_n.\] 
Therefore, $u\in L(G).$

Conversely, let $u=a_1a_2\ldots a_n\in L(G)$ and $u\not= \varepsilon$. 
Then there exists a derivation   
\[q_0\Longrightarrow a_1q_1\Longrightarrow a_1a_2q_2\Longrightarrow \ldots 
\Longrightarrow a_1a_2\ldots a_{n-1}q_{n-1}\Longrightarrow a_1a_2\ldots a_n \koz ,\] 
in which productions  
\[q_0\rightarrow a_1q_1,\;\; q_1\rightarrow a_2q_2,\;\; \ldots,\;\;q_{n-2}\rightarrow 
a_{n-1}q_{n-1},\;\; q_{n-1}\rightarrow a_n\] were used, which by definition means that 
in DFA  ${\mathrm A}$ there is a walk 
\[ q_0 \overset{a_1}{\longrightarrow} q_1 \overset{a_2}{\longrightarrow} q_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{n-1}}{\longrightarrow} q_{n-1} \overset{a_n}{\longrightarrow} q_n \koz ,\] 
and since $q_n$ is a final state,  $u\in L({\mathrm A})\setminus \{\varepsilon\} \koz .$

If the DFA accepts also the empty word $\varepsilon$, then in the above grammar we introduce a new start 
symbol $q_0'$ instead of  $q_0$, consider the new production $q'_0\rightarrow \varepsilon$ 
and for each production $q_0\rightarrow \alpha$  introduce also  $q'_0\rightarrow \alpha$. 
\end{biz}

\begin{figure}\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(65.00,19.00)
\put(10.00,6.00){\circle{10.00}}
\put(35.00,6.00){\circle{10.00}}
\put(60.00,6.00){\circle{10.00}}
\put(1.00,6.00){\vector(1,0){4.00}}
\put(15.00,6.00){\vector(1,0){15.00}}
\put(40.00,6.00){\vector(1,0){15.00}}
\put(10.00,9.00){\oval(8.00,14.00)[t]}
\put(60.00,9.00){\oval(8.00,14.00)[t]}
\put(14.00,11.00){\vector(0,-1){2.00}}
\put(64.00,11.00){\vector(0,-1){2.00}}
\put(10.00,19.00){\makebox(0,0)[cc]{$a$}}
\put(60.00,19.00){\makebox(0,0)[cc]{$a$}}
\put(22.00,8.00){\makebox(0,0)[cc]{$b$}}
\put(47.00,8.00){\makebox(0,0)[cc]{$b$}}
\put(10.00,6.00){\makebox(0,0)[cc]{$q_0$}}
\put(35.00,6.00){\makebox(0,0)[cc]{$q_1$}}
\put(60.00,6.00){\makebox(0,0)[cc]{$q_2$}}
\put(60.00,6.00){\circle{8.00}}
\end{picture}
\caption{DFA\abraindex{DFA} of the Example \ref{KZdetnyelv}.}
\label{KZ3ka4}
\end{figure}


\begin{pelda}\label{KZdetnyelv}
Let ${\mathrm A}=(\{q_0, q_1,q_2\}, \{a,b\}, E, \{q_0\}, \{q_2\})$ be a DFA, where  
$E=\big\{(q_0,a,q_0),$ $(q_0,b,q_1),(q_1,b,q_2),(q_2,a,q_2) \big\}$.
The corresponding transition table is

\medskip\begin{center}
\begin{tabular}{c|cc}
$\delta$   &    a    &   b \\ \hline
$q_0$      &  $\{q_0\}$  & $\{q_1\}$\\ 
$q_1$      &  $\emptyset$      & $\{q_2\}$ \\ 
$q_2$      &   $\{q_2\}$ &  $\emptyset$
\end{tabular}
\end{center}

\medskip
The transition graph of  ${\mathrm A}$ is in Fig. \ref{KZ3ka4}.
By Theorem  \ref{KZdetreg1} we define regular grammar $G=(\{q_0, q_1, q_2\}, \{a,b\}, P, q_0)$ 
with the productions in $P$

\centerline{$q_0\rightarrow aq_0\;|\; bq_1$, \qquad  $q_1\rightarrow bq_2\;|\; b$, 
\qquad  $q_2\rightarrow aq_2 \;|\; a$ \koz .}

\noindent One may prove that $L({\mathrm A})=\{ a^mbba^n\;|\; m\ge 0, n\ge 0\}$.
\end{pelda}

The method described in the proof of Theorem  \ref{KZdetreg1} easily can be given as an algorithm. 
The productions of regular grammar $G=(Q,\Sigma, P, q_0)$ 
obtained from the DFA ${\mathrm A}=(Q,\Sigma, E, \{q_0\}, F)$ can be determined by the following algorithm.

\begin{algN}{Regular-Grammar-from-DFA(A)}\inddef{Regular-Grammar@\textsc{Regular-Grammar-from-DFA}}
1 \' $P\leftarrow \emptyset$\\ 
2 \' \key{for} \= all $p\in Q$\\
3 \'           \> \key{do} \key{for} \= all $a\in \Sigma$\\
4 \'           \>      \> \key{do} \key{for} \= all $q\in Q$\\
5 \'           \>      \>    \> \key{do} \key{if} \= $(p,a,q)\in E$\\
6 \'           \>    \>  \>  \> \key{then} \= $P\leftarrow P\cup \{ p\rightarrow aq \}$ \\
7  \'          \>    \>  \>  \>  \> \key{if} \= $q\in F$\\ 
8  \'          \>    \>  \>  \> \> \> \key{then} $P\leftarrow P\cup \{ p\rightarrow a \}$ \\
9 \' \key{if} \= $q_0\in F$ \\
10 \'          \> \key{then} $P\leftarrow P\cup \{ q_0\rightarrow \varepsilon \}$ \\
11 \' \key{return} $G$ 
\end{algN}
It is easy to see that the running time of the algorithm is  $\Theta(n^2m),$ 
if the number of states is $n$ and the number of letter in alphabet is $m$. 
In lines  2--4 we can consider only one loop, if we use the elements of $E$. 
Then the worst case running time is  $\Theta (p)$, where $p$ is the number 
of transitions of DFA. This is also $O(n^2m)$, since all transitions are possible.  This algorithm is:

\begin{alg}{Regular-Grammar-from-Dfa'(A)}\inddef{Regular-Grammar-from@\textsc{Regular-Grammar-from-DFA'}}
1 \' $P\leftarrow \emptyset$\\ 
2 \' \key{for} \= all $(p,a,q)\in E$\\ 
3 \'            \>  \key{do} \= $P\leftarrow P\cup \{ p\rightarrow aq \}$ \\
4  \'           \>            \> \key{if} \= $q\in F$\\ 
5  \'           \>            \>           \> \key{then} $P\leftarrow P\cup \{ p\rightarrow a \}$ \\
6 \' \key{if} $q_0\in F$ \\
7 \'  \> \key{then} $P\leftarrow P\cup \{ q_0\rightarrow \varepsilon \}$ \\
8 \' \key{return} $G$

\end{alg}

\begin{tetel}\label{KZregnemdet}
If  $L=L(G)$ is a regular language, then one may construct an NFA that accepts language $L$.
\end{tetel}
\begin{biz}
   Let $G=(N,T,P,S)$ be the grammar which generates language $L$. Define NFA
 ${\mathrm A}=(Q, T, E, \{S\}, F)$: 

$\bullet$ $Q=N\cup \{Z\}$, where $Z\not\in N\cup T$ (i.e. $Z$ is a new symbol),

$\bullet$ For every production $A\rightarrow aB$, define transition  $(A, a, B)$ in $E$.

$\bullet$ For every production $A\rightarrow a$, define transition  $(A, a, Z)$ in  $E$.

$\bullet$ $F=\left\{ \begin{array}{ll}
               \{Z\} & \textrm{if production } 
               S\rightarrow \varepsilon\textrm{ does not occur in } G, \\
               \{Z,S\} & \textrm{if  production }  
              S\	\rightarrow \varepsilon \textrm{ occurs in } G.
\end{array} \right.$

\noindent Prove that  $L(G)=L(A)$.

\noindent Let $u=a_1a_2\ldots a_n\in L(G)$, $u\not= \varepsilon$. Then there is in $G$ a derivation of word $u$:

\smallskip
\centerline{$S\Longrightarrow a_1A_1\Longrightarrow a_1a_2A_2\Longrightarrow\ldots \Longrightarrow a_1a_2\ldots a_{n-1}A_{n-1}\Longrightarrow
a_1a_2\ldots a_n \koz .$}

\smallskip \noindent This derivation is based on productions

\smallskip
\centerline{$S\rightarrow a_1A_1,\;\; A_1\rightarrow a_2A_2, \;\;\ldots, \;\;A_{n-2}\rightarrow a_{n-1}A_{n-1},\;\;  A_{n-1}\rightarrow a_n \koz.$}

\smallskip\noindent Then, by the definition of the transitions of NFA  ${\mathrm A}$  there exists a walk  
\[ S \overset{a_1}{\longrightarrow} A_1 \overset{a_2}{\longrightarrow} A_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{n-1}}{\longrightarrow} A_{n-1} \overset{a_n}{\longrightarrow} Z, \; \;Z\in F \koz .\]
Thus, $u\in L({\mathrm A})$. 
If $\varepsilon\in L(G)$, there is production $S\rightarrow \varepsilon$, but in this case the initial state is also a final one, so  $\varepsilon \in L(A)$. Therefore, $L(G)\subseteq L(A)$.

Let now  $u=a_1a_2\ldots a_n\in L({\mathrm A})$.  
Then there exists a walk 
\[ S \overset{a_1}{\longrightarrow} A_1 \overset{a_2}{\longrightarrow} A_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{n-1}}{\longrightarrow} A_{n-1} \overset{a_n}{\longrightarrow} Z, \; \;Z\in F \koz .\]
If  $u$ is the empty word, then instead of $Z$ we have in the above formula $S$, 
which also is a final state. In other cases only $Z$ can be as last symbol.
Thus, in  $G$ there exist the productions 
 \[ S\rightarrow a_1A_1, \;\;A_1\rightarrow a_2A_2, \;\; \ldots,  \;\; A_{n-2}\rightarrow a_{n-1}A_{n-1},  
\;\; A_{n-1}\rightarrow a_n \koz ,\]
and there is the derivation 
\[S\Longrightarrow a_1A_1\Longrightarrow a_1a_2A_2\Longrightarrow\ldots \Longrightarrow a_1a_2\ldots a_{n-1}A_{n-1}\Longrightarrow
a_1a_2\ldots a_n \koz ,\]   thus,  $u\in L(G)$ and therefore  $L(A)\subseteq L(G)$.
\end{biz}

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(65.00,40.00)
\put(10.00,27.00){\circle{10.00}}
\put(35.00,27.00){\circle{10.00}}
\put(60.00,27.00){\circle{10.00}}
\put(1.00,27.00){\vector(1,0){4.00}}
\put(15.00,27.00){\vector(1,0){15.00}}
\put(40.00,27.00){\vector(1,0){15.00}}
\put(10.00,30.00){\oval(8.00,14.00)[t]}
\put(60.00,30.00){\oval(8.00,14.00)[t]}
\put(14.00,32.00){\vector(0,-1){2.00}}
\put(64.00,32.00){\vector(0,-1){2.00}}
\put(10.00,40.00){\makebox(0,0)[cc]{$a$}}
\put(60.00,40.00){\makebox(0,0)[cc]{$a$}}
\put(22.00,29.00){\makebox(0,0)[cc]{$b$}}
\put(47.00,29.00){\makebox(0,0)[cc]{$b$}}
\put(10.00,27.00){\makebox(0,0)[cc]{$S$}}
\put(35.00,27.00){\makebox(0,0)[cc]{$A$}}
\put(60.00,27.00){\makebox(0,0)[cc]{$B$}}
\put(60.00,7.00){\circle{10.00}}
\put(60.00,7.00){\circle{8.00}}
\put(60.00,7.00){\makebox(0,0)[cc]{$Z$}}
\put(49.00,17.00){\makebox(0,0)[cc]{$b$}}
\put(62.00,17.00){\makebox(0,0)[cc]{$a$}}
\put(60.00,22.00){\vector(0,-1){10.00}}
\put(38.00,23.00){\vector(4,-3){18.00}}
\end{picture}
\caption{NFA associated to grammar in Example  \ref{KZnyelvaut}.}
\label{KZ3ka5}
\end{figure}

\begin{pelda}\label{KZnyelvaut}
Let $G=(\{S,A,B\}, \{a,b\},$ $\{S\rightarrow aS, S\rightarrow bA, A\rightarrow bB, A\rightarrow b,$ 
$B\rightarrow aB, B\rightarrow a\}, S)$ be a regular grammar. The  NFA associated is $A=(\{S,A,B,Z\}, \{a,b\}, E, \{S\}, \{Z\})$, where
$E=\big \{ (S,a,S), (S,b,A), (A,b,B), (A,b,Z),$ 
$(B,a,B), (B,a,Z) \big\}$. The corresponding transition table is
\begin{center}
\begin{tabular}{c|cc}
$\delta$ &    $a$       &   $b$     \\ \hline
$S$      &  $\{S\}$     & $\{ A\}$     \\ 
$A$      &  $\emptyset$ & $\{B,Z\}$    \\ 
$B$      &   $\{B,Z\}$  &  $\emptyset$ \\
$E$      &   $\emptyset$ &  $\emptyset$
\end{tabular}
\end{center}

\noindent
The transition graph is in Fig. \ref{KZ3ka5}. This NFA can be simplified, 
states $B$ and $Z$ can be contracted in one final state.
\end{pelda}

Using the above theorem we define an algorithm which associate an NFA 
${\mathrm A}=(Q, T, E,$ $\{S\}, F)$ to a regular grammar $G=(N,T,P,S)$.

\begin{algN}{NFA-from-Regular-Grammar(A)}\inddef{nondet@\textsc{NFA-from-Regular-Grammar}}
1 \' $E\leftarrow \emptyset$\\
2 \' $Q\leftarrow N\cup \{Z\}$\\ 
3 \' \key{for} \= all $A\in N$\\
4 \'           \> \key{do} \key{for} \= all $a\in T$\\
5 \'           \>      \>  \key{do} \=  \key{if} \= $(A\rightarrow a)\in P$  \\
6 \'           \>      \>           \>         \> \key{then}  $E\leftarrow E\cup \{(A,a,Z)\}$ \\
7 \'           \>      \>           \> \key{for} \= all $B\in N$\\
8 \'           \>      \>           \>           \> \key{do} \key{if} \= $(A\rightarrow aB)\in P$  \\
9 \'           \>    \>  \>  \> \> \key{then} \= $E\leftarrow E\cup \{(A,a,B)\}$ \\\pushtabs
10 \' \key{if} \= $(S\rightarrow \varepsilon)\not\in P$  \\      
11 \'          \> \key{then}  \= $F\leftarrow \{Z\}$  \\
12 \'          \> \key{else}  \> $F\leftarrow \{Z,S\}$  \poptabs \\
13 \' \key{return} A
\end{algN}
As in the case of algorithm  \textsc{Regular-Grammar-from-DFA}, the running time is   
$\Theta(n^2m)$, where  $n$ is number of nonterminals and  $m$ the number of terminals. 
Loops in lines  3, 4 and  7 can be replaced by only one, which uses productions. 
The running time in this case is better and is equal to $\Theta(p)$, if  
 $p$ is the number of productions. This algorithm is:
 
\begin{algN}{NFA-from-Regular-Grammar'(A)}
\inddef{NFA-from-Regular@\textsc{NFA-from-Regular-Grammar'}}
1 \' $E\leftarrow \emptyset$\\
2 \' $Q\leftarrow N\cup \{Z\}$\\ 
3 \' \key{for} \= all $(A\rightarrow u)\in P$\\
4 \'           \>      \key{do} \=  \key{if} \= $u= a$  \\
5 \'           \>                \>          \> \key{then}   $E\leftarrow E\cup \{(A,a,Z)\}$ \\
6 \'           \>                \> \key{if} \= $u= aB$  \\
7 \'           \>                \>          \>  \key{then} \= $E\leftarrow E\cup \{(A,a,B)\}$ \\\pushtabs
8 \' \key{if} \= $(S\rightarrow \varepsilon)\not\in P$  \\      
9 \'          \> \key{then}  \= $F\leftarrow \{Z\}$  \\
10 \'          \> \key{else}  \> $F\leftarrow \{Z,S\}$  \poptabs \\
11 \' \key{return} A
\end{algN}
From Theorems \ref{KZnemdetdet},  \ref{KZdetreg1} and \ref{KZregnemdet} results that the 
class of regular languages coincides with the class of languages accepted by NFA's and also with class of languages accepted by DFA's. The result of these three theorems is illustrated in Fig.  \ref{KZ3ka3} and can be summarised also in the following theorem. 
\begin{tetel}
The following three class of  languages are the same:

$\bullet$ the class of regular languages,

$\bullet$ the class of languages accepted by DFA's,

$\bullet$ the class of languages accepted by NFA's.
\end{tetel}

\begin{figure}[t!]\centering
\hspace*{-0.5cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(92.00,36.00)
\put(18.00,30.00){\makebox(0,0)[cc]{{\small Regular grammars}}}
\put(74.00,32.00){\makebox(0,0)[cc]{{\small Nondeterministic}}}
\put(74.00,28.00){\makebox(0,0)[cc]{{\small finite automata}}}
\put(47.00,11.00){\makebox(0,0)[cc]{{\small Deterministic}}}
\put(47.00,7.00){\makebox(0,0)[cc]{{\small finite automata}}}
\put(18.00,30.00){\oval(36.00,12.00)[]}
\put(74.00,30.00){\oval(36.00,12.00)[]}
\put(47.00,9.00){\oval(36.00,12.00)[]}
\put(36.00,30.00){\vector(1,0){20.00}}
\put(74.00,24.00){\vector(-3,-2){14.00}}
\put(35.00,15.00){\vector(-2,1){17.00}}
\end{picture}
\caption{Relations between regular grammars and finite automata. To any regular grammar 
one may construct an NFA which accepts the language generated by that grammar. 
Any NFA can be transformed in an equivalent DFA. To any DFA one may construct 
a regular grammar which generates the language accepted by that DFA.}
\label{KZ3ka3} 
\end{figure}


\subsubsection{Operation on regular languages}\index{operations!on regular languages}

It is known (see Theorem \ref{KZzartsagi}) that the set ${\mathcal L}_3$ of regular 
languages is closed under the regular operations, that is if  $L_1, L_2$ are regular languages, 
then languages  $L_1\cup L_2$, $L_1L_2$ and $L_1^*$ are also regular.   
For regular languages are true also the following statements.

\textit{The complement of a regular language is also regular.}
This  is easy to prove using automata.  Let $L$ be a regular language and 
let ${\mathrm A}=(Q, \Sigma, E, \{q_0\}, F)$ be a DFA which accepts language $L$.  
It is easy to see that the DFA  $\overline{{\mathrm A}}=(Q, \Sigma, E, \{q_0\}, Q\setminus F)$ 
accepts language  $\overline{L}$. So, $\overline{L}$  is also regular. 

\textit{The intersection of two regular languages is also regular.} 
Since $L_1\cap L_2= \overline{\overline{L_1}\cup \overline{L_2}}$, 
the intersection is also regular.

\textit{The difference of two regular languages is also regular.}
Since $L_1\setminus L_2= L_1\cap \overline{L_2}$, the difference is also regular.


%\subsection{Finite automata with $\boldsymbol{{\varepsilon}}$-moves}
\subsection{Finite automata with special moves}


%\index{automaton!with $\varepsilon$-moves}\index{finite automaton!with $\varepsilon$-moves}


%\index{automaton!with $\epsilon$-moves}
A finite automaton with $\varepsilon$-moves (FA with $\varepsilon$-moves) extends NFA in such way that it may have transitions on the empty input $\varepsilon$, i.e. it may change a state without reading any input symbol.        
 In the case of a FA with $\varepsilon$-moves ${\mathrm A}= (Q, \Sigma, E, I,F)$ for the set of transitions it is true that $E\subseteq Q\times \big(\Sigma\cup \{\varepsilon\}\big) \times Q.$

The transition function of a FA with  $\varepsilon$-moves is:

\centerline{$ \delta: Q\times\big(\Sigma\cup \{\varepsilon\}\big)\rightarrow  {\mathcal P}(Q), \quad  
\delta(p,a)= \{ q \in Q \; | \; (p,a,q)\in E\} \koz . $}

The FA with $\varepsilon$-moves in Fig. \ref{KZ3ka1} accepts words of form $uvw$, where $u\in\{1\}^*, \; v\in \{0\}^*$ and  $w\in \{1\}^*$.
 
\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(65.00,19.00)
\put(10.00,6.00){\circle{10.00}}
\put(35.00,6.00){\circle{10.00}}
\put(60.00,6.00){\circle{10.00}}
\put(1.00,6.00){\vector(1,0){4.00}}
\put(15.00,6.00){\vector(1,0){15.00}}
\put(40.00,6.00){\vector(1,0){15.00}}
\put(10.00,9.00){\oval(8.00,14.00)[t]}
\put(35.00,9.00){\oval(8.00,14.00)[t]}
\put(60.00,9.00){\oval(8.00,14.00)[t]}
\put(14.00,11.00){\vector(0,-1){2.00}}
\put(39.00,11.00){\vector(0,-1){2.00}}
\put(64.00,11.00){\vector(0,-1){2.00}}
\put(10.00,19.00){\makebox(0,0)[cc]{1}}
\put(35.00,19.00){\makebox(0,0)[cc]{0}}
\put(60.00,19.00){\makebox(0,0)[cc]{1}}
\put(22.00,8.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(47.00,8.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(10.00,6.00){\makebox(0,0)[cc]{$q_0$}}
\put(35.00,6.00){\makebox(0,0)[cc]{$q_1$}}
\put(60.00,6.00){\makebox(0,0)[cc]{$q_2$}}
\put(60.00,6.00){\circle{8.00}}
\end{picture}
\caption{Finite automata $\epsilon$-moves.}
\label{KZ3ka1}
\end{figure}

\begin{tetel}
To any FA with $\varepsilon$-moves one may construct an equivalent NFA (without $\varepsilon$-moves).
\end{tetel}
Let  ${\mathrm A}=(Q,\Sigma, E,I,F)$ be an FA with $\varepsilon$-moves and 
we construct an equivalent NFA  $\overline{{\mathrm A}}=(Q,\Sigma, \overline{E},I,\overline{F})$.
 The following algorithm determines sets $\overline{F}$ and $\overline{E}$.
 
 For a state  $q$ denote by $\Lambda (q)$  the set of states (including even $q$) in which one may go from $q$ using 
 $\varepsilon$-moves only. This may be extended also to sets  
\[\Lambda(S)= \displaystyle\mathop\bigcup_{q\in S}\Lambda(q), \quad \forall S\subseteq Q \koz .\]
Clearly, for all $q\in Q$ and $S\subseteq Q$  both $\Lambda(q)$ and $\Lambda(S)$ 
may be computed. Suppose in the sequel that these are given.

The following algorithm determine the transitions using the transition function $\overline{\delta}$, 
which is defined in line 5. 

If  $|Q|=n$ and $|\Sigma|=m,$, then lines  2--6 show that the running time in worst case is  $O(n^2m).$ 
 
\begin{alg}{Eliminate-Epsilon-Moves(A)}\inddef{elimin@\textsc{Eliminate-Epsilon-Moves}}
1 \'  $\overline{F} \leftarrow   F\cup \{q\in I \;|\; \Lambda(q)\cap F\not=\emptyset\} $ \\
2 \'  \key{for} \= all $q\in Q$\\
3 \'           \> \key{do} \key{for} \=  all $a\in \Sigma$\\
4  \'  \>              \>  \key{do} \=  $\Delta\leftarrow\displaystyle\mathop\bigcup_{p\in\Lambda(q)}\delta(p,a)$\\ 
5  \'  \>               \>          \> $\overline{\delta}(q,a)\leftarrow\Delta\cup\left(\displaystyle\mathop\bigcup_{p\in\Delta}\Lambda(p)\right)$ \\
6 \'  $\overline{E} \leftarrow \displaystyle{\big\{(p,a,q), \;|\;  p,q\in Q, \; a\in \Sigma, 
q\in \overline{\delta}(p,a) \big\}}$ \\
7 \' \key{return} $\overline{\mathrm A}$
\end{alg}

 
\begin{pelda}
Consider the FA with $\varepsilon$-moves in Fig. \ref{KZ3ka1}. The corresponding transition table is:

\medskip
\begin{center}
\begin{tabular}{c|ccc}
 $\delta$ &  0           &  1           & $\varepsilon$ \\ \hline
 $q_0 $   & $\emptyset$  &  $\{q_0\}$   & $\{q_1\}$ \\
 $q_1 $   & $\{q_1\}$    &  $\emptyset$ & $\{q_2\}$ \\
 $q_2 $   & $\emptyset$  &  $\{q_2\}$   & $\emptyset$
\end{tabular}
\end{center}

\medskip\noindent  Apply algorithm  \textsc{Eliminate-Epsilon-Moves}.

\noindent $\Lambda(q_0)=\{ q_0, q_1,q_2\}$, $\;\Lambda(q_1)=\{q_1,q_2\}$, $\;\Lambda(q_2)=\{q_2\}$

\noindent $\Lambda(I)=\Lambda(q_0)$, and its intersection with  $F$ is not empty, thus  
$\overline{F}=F\cup \{q_0\}= \{q_0,q_2\}$.

\noindent $(q_0,0):$   

$\Delta= \delta(q_0,0)\cup \delta(q_1,0)\cup \delta(q_2, 0)=\{q_1\}$,
$\;\;\{q_1\}\cup \Lambda(q_1)=\{q_1,q_2\}$

$\overline{\delta}(q_0,0)=\{q_1,q_2\}$.

\noindent $(q_0,1):$

$\Delta= \delta(q_0,1)\cup \delta(q_1,1)\cup \delta(q_2, 1)=\{q_0,q_2\}$, 
$\;\;\{q_0,q_2\}\cup \left( \Lambda(q_0)\cup \Lambda(q_2) \right)= \{q_0,q_1,q_2\}$

$\overline{\delta}(q_0,1)= \{q_0,q_1,q_2\}$


\noindent $(q_1,0):$ 

$\Delta= \delta(q_1,0)\cup \delta(q_2,0)=\{q_1\}$, $\;\;\{q_1\} \cup \Lambda(q_1)=\{q_1,q_2\}$

$\overline{\delta}(q_1,0)=\{q_1,q_2\}$


\noindent $(q_1,1):$ 

$\Delta= \delta(q_1,1)\cup \delta(q_2,1)=\{q_2\}$, $\;\;\{q_2\}\cup \Lambda(q_2)=\{q_2\}$

$\overline{\delta}(q_1,1)=\{q_2\}$


\noindent $(q_2,0):$ $\Delta= \delta(q_2,0)=\emptyset$

$\overline{\delta}(q_2,0)=\emptyset$


\noindent $(q_2,1):$  

$\Delta= \delta(q_2,1)=\{q_2\}$, $\;\;\{q_2\}\cup \Lambda(q_2)=\{q_2\}$

$\overline{\delta}(q_2,1)=\{q_2\} .$

\smallskip
\noindent The transition table of NFA  $\overline{\mathrm A}$ is:

\medskip\begin{center}
\begin{tabular}{c|cc}
 $\overline{\delta}$ &  0             &  1                \\ \hline
 $q_0 $        & $\{q_1,q_2\}$  &  $\{q_0,q_1,q_2\}$   \\
 $q_1 $        & $\{q_1,q_2\}$  &  $\{q_2\}$  \\
 $q_2 $        & $\emptyset$  &  $\{q_2\}$   
\end{tabular}
\end{center}
\noindent and the transition graph is in Fig. \ref{KZ3ka2}.
\end{pelda}

\begin{figure}\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(65.00,34.00)
\put(10.00,21.00){\circle{10.00}}
\put(35.00,21.00){\circle{10.00}}
\put(60.00,21.00){\circle{10.00}}
\put(1.00,21.00){\vector(1,0){4.00}}
\put(15.00,21.00){\vector(1,0){15.00}}
\put(40.00,21.00){\vector(1,0){15.00}}
\put(10.00,24.00){\oval(8.00,14.00)[t]}
\put(35.00,24.00){\oval(8.00,14.00)[t]}
\put(60.00,24.00){\oval(8.00,14.00)[t]}
\put(14.00,26.00){\vector(0,-1){2.00}}
\put(39.00,26.00){\vector(0,-1){2.00}}
\put(64.00,26.00){\vector(0,-1){2.00}}
\put(10.00,34.00){\makebox(0,0)[cc]{1}}
\put(35.00,34.00){\makebox(0,0)[cc]{0}}
\put(60.00,34.00){\makebox(0,0)[cc]{1}}
\put(22.00,23.00){\makebox(0,0)[cc]{0,1}}
\put(47.00,23.00){\makebox(0,0)[cc]{0,1}}
\put(10.00,21.00){\makebox(0,0)[cc]{$q_0$}}
\put(35.00,21.00){\makebox(0,0)[cc]{$q_1$}}
\put(60.00,21.00){\makebox(0,0)[cc]{$q_2$}}
\put(60.00,21.00){\circle{8.00}}
\put(10.00,21.00){\circle{8.00}}
\bezier{236}(10.00,16.00)(35.00,0.00)(60.00,16.00)
\put(59.00,15.35){\vector(2,1){1.00}}
\put(35.00,10.00){\makebox(0,0)[cc]{0,1}}
\end{picture}
\caption{NFA equivalent to FA with $\varepsilon$-moves given in Fig. \ref{KZ3ka1}.}
\label{KZ3ka2}
\end{figure}

Define regular operations on NFA: union, product and iteration.  The result will be an FA with  $\varepsilon$-moves.

Operation will be given also by diagrams. An NFA is given as in Fig.  \ref{KZ3ka6}(a). Initial states are represented by a circle with an arrow, final states by a double circle.

\begin{figure}[t!] 
\hspace*{0.5cm} 
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(35.00,35.00)
\emline{10.00}{20.00}{1}{35.00}{20.00}{2}
\emline{35.00}{20.00}{3}{35.00}{35.00}{4}
\emline{35.00}{35.00}{5}{10.00}{35.00}{6}
\emline{10.00}{20.00}{7}{10.00}{35.00}{8}
\put(15.00,27.00){\circle{4.00}}
\put(30.00,27.00){\circle{4.00}}
\put(30.00,27.00){\circle{2.00}}
\put(11.00,27.00){\vector(1,0){2.00}}
\put(22.00,2.00){\makebox(0,0)[cc]{(a)}}
\end{picture}
\quad\quad
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(56.00,58.00)
\emline{26.00}{34.00}{1}{51.00}{34.00}{2}
\emline{51.00}{34.00}{3}{51.00}{49.00}{4}
\emline{51.00}{49.00}{5}{26.00}{49.00}{6}
\emline{26.00}{34.00}{7}{26.00}{49.00}{8}
\put(31.00,41.00){\circle{4.00}}
\put(46.00,41.00){\circle{4.00}}
\put(46.00,41.00){\circle{2.00}}
\emline{26.00}{12.00}{9}{51.00}{12.00}{10}
\emline{51.00}{12.00}{11}{51.00}{27.00}{12}
\emline{51.00}{27.00}{13}{26.00}{27.00}{14}
\emline{26.00}{12.00}{15}{26.00}{27.00}{16}
\put(31.00,19.00){\circle{4.00}}
\put(46.00,19.00){\circle{4.00}}
\put(46.00,19.00){\circle{2.00}}
\emline{1.00}{7.00}{17}{56.00}{7.00}{18}
\emline{56.00}{7.00}{19}{56.00}{55.00}{20}
\emline{56.00}{55.00}{21}{1.00}{55.00}{22}
\emline{1.00}{55.00}{23}{1.00}{7.00}{24}
\put(10.00,31.00){\circle{4.00}}
\put(5.00,31.00){\vector(1,0){3.00}}
\bezier{100}(10.00,33.00)(17.00,44.00)(29.00,41.00)
\bezier{104}(10.00,29.00)(18.00,17.00)(29.00,19.00)
\put(28.00,41.20){\vector(1,0){1.00}}
\put(28.00,18.80){\vector(1,0){1.00}}
\put(17.00,42.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(17.00,18.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(28.00,51.00){\makebox(0,0)[cc]{\small $A_1$}}
\put(28.00,29.00){\makebox(0,0)[cc]{\small $A_2$}}
\put(7.00,58.00){\makebox(0,0)[cc]{\small $A_1\cup A_2$}}
\put(28.00,2.00){\makebox(0,0)[cc]{(b)}}
\end{picture}
\caption{\textbf{(a)} Representation of an NFA. Initial states are represented by a circle with an arrow, final states by a double circle. \textbf{(b)} Union of two NFA's.}
\label{KZ3ka6}
\end{figure}


Let  ${\mathrm A}_1=(Q_1 ,\Sigma_1, E_1, I_1, F_1)$ and 
 ${\mathrm A}_2=(Q_2,\Sigma_2, E_2, I_2, F_2)$ be NFA.  The result of any operation is a FA with $\varepsilon$-moves
${\mathrm A}=(Q, \Sigma, E, I, F)$.  Suppose that  $Q_1\cap Q_2=\emptyset$ always. If not,  we can rename the elements of any set of states. 


\textit{Union.} ${\mathrm A}={\mathrm A}_1\cup {\mathrm A}_2$, where

\hspace*{1.5 cm} $Q=Q_1\cup Q_2\cup \{q_0\}$ \koz ,

\hspace*{1.5 cm} $\Sigma=\Sigma_1\cup \Sigma_2$ \koz,

\hspace*{1.5 cm} $I=\{q_0\}$ \koz,

\hspace*{1.5 cm} $F=F_1\cup F_2$ \koz,

\hspace*{1.5 cm} $E=E_1\cup E_2 \cup \displaystyle\bigcup_{q\in I_1\cup I_2} \big\{ (q_0,\varepsilon, q) \big\}$ \koz.

\medskip
For the result of the union see Fig.  \ref{KZ3ka6}(b). The result is the same if instead of a single 
initial state we choose as set of initial states the union $I_1\cup I_2$. 
In this case the result automaton will be without 
 $\varepsilon$-moves. By the definition it is easy to see that  $L(A_1\cup A_2)=L(A_1)\cup L(A_2)$.

\textit{Product.} ${\mathrm A}={\mathrm A}_1\cdot {\mathrm A}_2$, where

\hspace*{1.5 cm} $Q=Q_1\cup Q_2$,

\hspace*{1.5 cm} $\Sigma=\Sigma_1\cup \Sigma_2$,

\hspace*{1.5 cm} $F= F_2$,

\hspace*{1.5 cm} $I= I_1$,

\hspace*{1.5 cm} $E=E_1\cup E_2 \cup \displaystyle\bigcup_{{\scriptsize \begin{array}{c}p\in F_1\\ q\in I_2 \end{array}}}
 \big\{(p,\varepsilon, q)\big\} \koz .$
 
 For the result automaton see Fig.  \ref{KZ3ka8}(a). Here also $L(A_1\cdot A_2)=L(A_1)L(A_2)$.
 
 \noindent\begin{figure}[t!] 
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(72.00,37.00)
\emline{8.00}{11.00}{1}{33.00}{11.00}{2}
\emline{33.00}{11.00}{3}{33.00}{26.00}{4}
\emline{33.00}{26.00}{5}{8.00}{26.00}{6}
\emline{8.00}{11.00}{7}{8.00}{26.00}{8}
\put(15.00,18.00){\circle{4.00}}
\put(28.00,18.00){\circle{4.00}}
\put(28.00,18.00){\circle{2.00}}
\emline{42.00}{11.00}{9}{67.00}{11.00}{10}
\emline{67.00}{11.00}{11}{67.00}{26.00}{12}
\emline{67.00}{26.00}{13}{42.00}{26.00}{14}
\emline{42.00}{11.00}{15}{42.00}{26.00}{16}
\put(47.00,18.00){\circle{4.00}}
\put(62.00,18.00){\circle{4.00}}
\put(62.00,18.00){\circle{2.00}}
\put(10.00,18.00){\vector(1,0){3.00}}
\put(10.00,28.00){\makebox(0,0)[cc]{\small $A_1$}}
\put(44.00,28.00){\makebox(0,0)[cc]{\small $A_2$}}
\put(38.00,25.00){\makebox(0,0)[cc]{$\varepsilon$}}
\emline{2.00}{34.00}{17}{2.00}{7.00}{18}
\put(2.00,37.00){\makebox(0,0)[lc]{\small $A_1\cdot A_2$ }}
\put(44.50,19.70){\vector(1,-1){1.00}}
\bezier{84}(29.50,19.00)(38.00,26.00)(45.00,19.00)
\emline{72.00}{34.00}{19}{72.00}{7.00}{20}
\emline{72.00}{7.00}{21}{2.00}{7.00}{22}
\emline{2.00}{34.00}{23}{72.00}{34.00}{24}
\put(37.00,2.00){\makebox(0,0)[cc]{(a)}}
\end{picture}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(50.00,35.00)
\emline{22.00}{12.00}{1}{47.00}{12.00}{2}
\emline{47.00}{12.00}{3}{47.00}{27.00}{4}
\emline{47.00}{27.00}{5}{22.00}{27.00}{6}
\emline{22.00}{12.00}{7}{22.00}{27.00}{8}
\put(27.00,19.00){\circle{4.00}}
\put(42.00,19.00){\circle{4.00}}
\put(42.00,19.00){\circle{2.00}}
\put(10.00,19.00){\circle{4.00}}
\put(5.00,19.00){\vector(1,0){3.00}}
\put(24.00,19.00){\vector(1,0){1.00}}
\put(24.00,29.00){\makebox(0,0)[cc]{\small $A_1$}}
\put(10.00,19.00){\circle{2.00}}
\put(16.00,21.00){\makebox(0,0)[cc]{$\varepsilon$}}
\bezier{68}(27.00,21.00)(34.00,25.00)(42.00,21.00)
\put(28.20,21.75){\vector(-1,-1){1.00}}
\put(34.00,25.00){\makebox(0,0)[cc]{$\varepsilon$}}
\emline{3.00}{7.00}{9}{3.00}{32.00}{10}
\put(3.00,35.00){\makebox(0,0)[lc]{$A_1^*$}}
\emline{12.00}{19.00}{11}{24.00}{19.00}{12}
\emline{50.00}{32.00}{13}{50.00}{7.00}{14}
\emline{50.00}{32.00}{15}{3.00}{32.00}{16}
\emline{3.00}{7.00}{17}{50.00}{7.00}{18}
\put(26.00,2.00){\makebox(0,0)[cc]{(b)}}
\end{picture}
\caption{\textbf{(a)} Product of two FA.  \textbf{(b)} Iteration of an FA.}
\label{KZ3ka8}
\end{figure}

\textit{Iteration.} ${\mathrm A}={\mathrm A_1}^*,$ where

\hspace*{1.5 cm} $Q=Q_1\cup \{q_0\},$

\hspace*{1.5 cm} $\Sigma=\Sigma_1,$

\hspace*{1.5 cm} $F= F_1\cup \{q_0\},$

\hspace*{1.5 cm} $I=\{q_0\}$

\hspace*{1.5 cm} $E= E_1\cup \displaystyle\bigcup_{p\in I_1}\big\{(q_0,\varepsilon,p ) \big \}   
   \cup \displaystyle\bigcup_{{\scriptsize \begin{array}{c}q\in F_1\\ p\in I_1 
    \end{array}} }\big\{(q,\varepsilon,p ) \big \} \koz .$

\medskip \vspace{-1mm}
The iteration of an FA can be seen in Fig.   \ref{KZ3ka8}(b). 
For this operation it is also true that  $L(A_1^*)=\big(L(A_1)\big)^*$.

The definition of these tree operations proves again that regular languages are closed under the regular operations.

\vspace{-1mm}
\subsection{Minimization of finite automata}
\index{finite automata!minimization}\index{automata!minimization}
A  DFA  ${\mathrm A}=(Q,\Sigma, E,\{q_0\},F)$ is called  \ki{minimum state automaton} if for any equivalent complete DFA  ${\mathrm A}'=(Q',\Sigma, E',\{q'_0\},F')$ it is true that $|Q|\le |Q'|$. 
We give an algorithm which builds for any complete DFA an equivalent minimum state automaton. 

\begin{figure}\centering
\begin{minipage}[b]{6.2cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(61.00,72.00)
\put(7.00,21.00){\circle{8.00}}
\put(32.00,21.00){\circle{8.00}}
\put(57.00,21.00){\circle{8.00}}
\put(7.00,56.00){\circle{8.00}}
\put(32.00,56.00){\circle{8.00}}
\put(57.00,56.00){\circle{8.00}}
\put(0.00,56.00){\vector(1,0){3.00}}
\put(11.00,56.00){\vector(1,0){17.00}}
\put(36.00,56.00){\vector(1,0){17.00}}
\put(10.00,53.00){\vector(2,-3){19.33}}
\put(32.00,52.00){\vector(0,-1){27.00}}
\put(57.00,25.00){\vector(0,1){27.00}}
\put(53.00,21.00){\vector(-1,0){17.00}}
\put(27.00,22.60){\vector(2,-1){1.00}}
\put(11.50,17.70){\vector(-1,1){1.00}}
\put(56.00,16.20){\vector(1,1){1.00}}
\put(31.50,17.50){\oval(5.00,7.00)[b]}
\put(34.00,16.50){\vector(0,1){1.00}}
\put(57.00,59.50){\oval(4.00,10.00)[t]}
\put(59.00,61.00){\vector(0,-1){1.00}}
\put(10.30,59.90){\vector(-2,-1){1.00}}
\put(7.00,56.00){\makebox(0,0)[cc]{$q_0$}}
\put(32.00,56.00){\makebox(0,0)[cc]{$q_1$}}
\put(57.00,56.00){\makebox(0,0)[cc]{$q_2$}}
\put(7.00,21.00){\makebox(0,0)[cc]{$q_3$}}
\put(32.00,21.00){\makebox(0,0)[cc]{$q_4$}}
\put(57.00,21.00){\makebox(0,0)[cc]{$q_5$}}
\put(32.00,72.00){\makebox(0,0)[cc]{0}}
\put(57.00,67.00){\makebox(0,0)[cc]{1}}
\put(57.00,56.00){\circle{6.00}}
\put(20.00,58.00){\makebox(0,0)[cc]{0}}
\put(43.00,58.00){\makebox(0,0)[cc]{1}}
\put(20.00,41.00){\makebox(0,0)[cc]{1}}
\put(35.00,41.00){\makebox(0,0)[cc]{0}}
\put(59.00,41.00){\makebox(0,0)[cc]{1}}
\put(19.00,27.00){\makebox(0,0)[cc]{1}}
\put(19.00,18.00){\makebox(0,0)[cc]{1}}
\put(44.00,23.00){\makebox(0,0)[cc]{0}}
\put(33.00,12.00){\makebox(0,0)[cc]{0}}
\put(33.00,5.00){\makebox(0,0)[cc]{0}}
\bezier{200}(10.50,60.00)(32.00,72.00)(55.00,60.00)
\bezier{240}(8.00,17.00)(33.00,0.00)(56.70,16.80)
\bezier{88}(11.00,18.00)(18.00,12.00)(29.00,18.00)
\bezier{84}(28.00,22.00)(19.00,28.00)(11.00,22.00)
\end{picture}
\end{minipage}
\begin{minipage}[b]{5.3cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(120.00,56.00)
\emline{5.00}{2.00}{1}{45.00}{2.00}{2}
\emline{5.00}{2.00}{3}{5.00}{12.00}{4}
\emline{5.00}{12.00}{5}{45.00}{12.00}{6}
\emline{45.00}{12.00}{7}{45.00}{2.00}{8}
\emline{35.00}{2.00}{9}{35.00}{12.00}{10}
\emline{25.00}{2.00}{11}{25.00}{12.00}{12}
\emline{15.00}{12.00}{13}{15.00}{2.00}{14}
\emline{45.00}{2.00}{15}{55.00}{2.00}{16}
\emline{55.00}{2.00}{17}{55.00}{12.00}{18}
\emline{55.00}{12.00}{19}{45.00}{12.00}{20}
\emline{15.00}{12.00}{21}{15.00}{22.00}{22}
\emline{15.00}{22.00}{23}{55.00}{22.00}{24}
\emline{55.00}{22.00}{25}{55.00}{12.00}{26}
\emline{25.00}{22.00}{27}{25.00}{32.00}{28}
\emline{25.00}{32.00}{29}{55.00}{32.00}{30}
\emline{55.00}{32.00}{31}{55.00}{22.00}{32}
\emline{55.00}{32.00}{33}{55.00}{42.00}{34}
\emline{55.00}{42.00}{35}{35.00}{42.00}{36}
\emline{35.00}{42.00}{37}{35.00}{32.00}{38}
\emline{45.00}{42.00}{39}{45.00}{52.00}{40}
\emline{45.00}{52.00}{41}{55.00}{52.00}{42}
\emline{55.00}{52.00}{43}{55.00}{42.00}{44}
\emline{45.00}{42.00}{45}{45.00}{12.00}{46}
\emline{35.00}{12.00}{47}{35.00}{42.00}{48}
\emline{25.00}{22.00}{49}{25.00}{12.00}{50}
\put(2.00,7.00){\makebox(0,0)[cc]{$q_0$}}
\put(10.00,17.00){\makebox(0,0)[cc]{$q_1$}}
\put(20.00,27.00){\makebox(0,0)[cc]{$q_2$}}
\put(30.00,37.00){\makebox(0,0)[cc]{$q_3$}}
\put(40.00,47.00){\makebox(0,0)[cc]{$q_4$}}
\put(50.00,56.00){\makebox(0,0)[cc]{$q_5$}}
\put(10.00,7.00){\makebox(0,0)[cc]{*}}
\put(20.00,7.00){\makebox(0,0)[cc]{*}}
\put(40.00,7.00){\makebox(0,0)[cc]{*}}
\put(50.00,7.00){\makebox(0,0)[cc]{*}}
\put(40.00,17.00){\makebox(0,0)[cc]{*}}
\put(30.00,17.00){\makebox(0,0)[cc]{*}}
\put(20.00,17.00){\makebox(0,0)[cc]{*}}
\put(30.00,27.00){\makebox(0,0)[cc]{*}}
\put(40.00,27.00){\makebox(0,0)[cc]{*}}
\put(50.00,27.00){\makebox(0,0)[cc]{*}}
\put(40.00,37.00){\makebox(0,0)[cc]{*}}
\put(50.00,37.00){\makebox(0,0)[cc]{*}}
\put(50.00,47.00){\makebox(0,0)[cc]{*}}
\end{picture}
\end{minipage}\caption{Minimization of DFA.\abraindex{DFA}}
\label{KZ3ka10}
\end{figure}

States $p$ and $q$ of an DFA  ${\mathrm A}=(Q,\Sigma, E,\{q_0\},F)$ are  \ki{equivalent}\inddef{equivalent states}  if for arbitrary word  $u$  we reach from both  either final  or nonfinal states, that is

\smallskip
$p\equiv q\;$ if for any word  $u\in\Sigma^*$   $\;\left\{ 
\begin{array}{l}
   p \overset{u}{\longrightarrow} r, \; r\in F  \; \textrm{ and } \; q \overset{u}{\longrightarrow} s, \; s \in F \;\; \textrm{ or} \\
 p \overset{u}{\longrightarrow} r, \;  r\not\in F  \;\textrm{ and } \;  q \overset{u}{\longrightarrow} s, \; s \not\in F \koz .
\end{array}\right.$

\smallskip If two states are not equivalent, then they are distinguishable.
In the following algorithm the distinguishable states will be marked by a star, and equivalent states will be 
merged. The algorithm will associate list of pair of states with some pair of states expecting a later marking by a star, that is if we mark a pair of states by a star, then all pairs on the associated list will be also marked by a star.
The algorithm is given for DFA without inaccessible states. The used DFA is complete, so 
 $\delta(p,a)$ contains exact one element, function $elem$ defined on page \pageref{egyelemu}, which gives the unique element of the set, will be also used here.

\begin{alg}{Automaton-Minimization(${\mathrm A}$)}\inddef{Automata-min@\textsc{Automaton-Minimization}}
1 \' mark with a star all pairs of states $\big\{p,q\big\}$  for which\\ 
  \'  $p\in F$ and  $q\not\in F$ or  $p\not\in F$ and  $q\in F$ \\
2 \' associate an empty list with each unmarked pair  $\big\{p,q\big\}$ \\ 
3  \' \key{for} \= all unmarked pair of states $\big\{p,q\big\}$ and for all symbol $a\in \Sigma$ \\
\'              \> examine pairs of states $\big\{elem\big(\delta(p,a)\big), elem\big(\delta(q,a)\big)\big\}$\\ 
\'               \>  \key{if} \= any of these pairs is marked,   \\
\'               \>           \> \key{then} \= mark also pair $\big\{p,q\big\}$ with all the elements on the list before   \\
\'               \>           \>            \> associated with pair $\big\{p,q\big\}$\\ 
\'               \>           \> \key{else} \= \key{if} \= all the above pairs are unmarked  \\
\'               \>           \>            \>          \> \key{then} \= put pair $\{p,q\}$ on each list associated with pairs \\
\'               \>           \>            \>          \>            \> $\big\{elem\big(\delta(p,a)\big), elem\big(\delta(q,a)\big)\big\},$  
              unless  $\delta(p,a)= \delta(q,a)$\\
4 \' merge all unmarked (equivalent) pairs 
\end{alg}



\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(95.00,65.00)
\put(10.00,39.00){\circle{11.00}}
\put(45.00,39.00){\circle{11.00}}
\put(80.00,39.00){\circle{11.00}}
\put(45.00,8.00){\circle{11.00}}
\put(0.20,39.00){\vector(1,0){4.00}}
\put(80.00,39.00){\circle{9.00}}
\put(15.70,39.00){\vector(1,0){23.50}}
\put(50.80,39.00){\vector(1,0){23.50}}
\put(45.00,33.20){\vector(0,-1){19.50}}
\bezier{324}(9.80,44.80)(45.00,65.00)(80.20,44.80)
\bezier{176}(42.00,13.00)(35.00,36.00)(15.00,36.00)
\bezier{196}(10.00,33.00)(16.00,8.00)(39.00,8.00)
\put(85.00,38.50){\oval(14.00,7.00)[r]}
\put(86.00,35.00){\vector(-1,0){2.00}}
\put(49.50,7.00){\oval(13.00,8.00)[r]}
\put(50.00,3.00){\vector(-1,0){2.00}}
\put(10.00,39.00){\makebox(0,0)[cc]{$q_0q_3$}}
\put(45.00,39.00){\makebox(0,0)[cc]{$q_1q_5$}}
\put(80.00,39.00){\makebox(0,0)[cc]{$q_2$}}
\put(45.00,8.00){\makebox(0,0)[cc]{$q_4$}}
\put(45.00,58.00){\makebox(0,0)[cc]{0}}
\put(28.00,41.00){\makebox(0,0)[cc]{0}}
\put(62.00,41.00){\makebox(0,0)[cc]{1}}
\put(95.00,39.00){\makebox(0,0)[cc]{1}}
\put(32.00,26.00){\makebox(0,0)[cc]{1}}
\put(21.00,16.00){\makebox(0,0)[cc]{1}}
\put(47.00,25.00){\makebox(0,0)[cc]{0}}
\put(59.00,7.00){\makebox(0,0)[cc]{0}}
\put(15.80,36.00){\vector(-1,0){1.00}}
\put(38.00,8.00){\vector(1,0){1.00}}
\put(11.80,45.80){\vector(-2,-1){2.00}}
\end{picture}
\caption{Minimum automaton equivalent with DFA in Fig. \ref{KZ3ka10}.}
\label{KZ3ka11}
\end{figure}

After finishing the algorithm, if a cell of the table does not contain a star, then the  states  corresponding to its row and column index, are equivalent and may be merged. Merging states is continued until it is possible.
We can say that the equivalence relation decomposes the set of states in equivalence classes, and the states in such a class may be all merged.

\emph{Remark.} The above algorithm can be used also in the case of an 
DFA which is not complete, that is there are states for which does not exist transition. 
Then a pair $\big\{\emptyset, \{q\}\big\}$ may occur, and if  $q$ is a final state, consider this pair marked.

\begin{pelda} Let be the DFA in Fig. \ref{KZ3ka10}. We will use a table for marking pairs with a star. 
Marking pair $\{p,q\}$ means putting a star in the cell corresponding to row $p$ and column 
$q$ (or row  $q$ and column $p$).


First we mark pairs $\{q_2,q_0\}$, $\{q_2,q_1\}$, $\{q_2,q_3\}$, $\{q_2,q_4\}$ and $\{q_2,q_5\}$ 
(because $q_2$ is the single final state). Then consider all unmarked pairs and 
examine them as  the algorithm requires.
Let us begin with  pair $\{q_0,q_1\}$. Associate with it pairs  
$\{elem\big(\delta(q_0,0)\big), elem\big(\delta(q_1,0)\big)\}, \{elem\big(\delta(q_0,1)\big),$ 
$elem\big(\delta(q_1,1)\big)\}$, that is $\{q_1,q_4\}$, $\{q_4,q_2\}$.  
Because pair $\{q_4,q_2\}$ is already marked, mark also pair  $\{q_0,q_1\}$.

In the case of pair $\{q_0,q_3\}$ the new pairs are $\{q_1,q_5\}$ and $\{q_4,q_4\}$. 
With pair $\{q_1,q_5\}$ associate pair $\{q_0,q_3\}$ on a list, that is
\[\{q_1,q_5\} \longrightarrow \{q_0,q_3\} \koz .\]
Now continuing with $\{q_1,q_5\}$ one obtain pairs  $\{q_4,q_4\}$ and  $\{q_2,q_2\}$, 
with which nothing are associated by algorithm.

Continue with pair $\{q_0,q_4\}$. The associated pairs are  $\{q_1,q_4\}$ and $\{q_4,q_3\}$. 
None of them are marked, so associate with them on a list pair  $\{q_0,q_4\}$, that is 
\[\{q_1,q_4\} \longrightarrow \{q_0,q_4\}, \qquad \{q_4,q_3\} \longrightarrow \{q_0,q_4\} \koz .\]
Now continuing with  $\{q_1,q_4\}$ we get the pairs  $\{q_4,q_4\}$ and $\{q_2,q_3\}$, 
and because this latter is marked we mark pair $\{q_1,q_4\}$ and also pair  $\{q_0,q_4\}$ associated to it on a list. 
Continuing we will get the table in Fig.  \ref{KZ3ka10}, that is we get that $q_0\equiv q_3$ 
and  $q_1\equiv q_5$. After merging them  we get an equivalent minimum state automaton (see Fig. \ref{KZ3ka11}).
\end{pelda}


\subsection{Pumping lemma for regular languages}\index{pumping lemma!for regular languages}
The following theorem, called  \textit{pumping lemma} for historical reasons,  
may be efficiently used to prove that a language is not regular. It is a sufficient condition for a regular language. 

\begin{tetel}[pumping lemma] \label{KZiteracios} For any regular language $L$ there exists a natural number 
 $n\ge 1$  (depending only on $L$), such that any word $u$ of  $L$  with length at least  $n$ 
may be written as $u=xyz$ such that 

(1) $|xy|\le n,$ 

(2) $|y|\ge 1,$

(3) $xy^iz \in L$ for all $i=0,1,2,\ldots $.
\end{tetel}
\begin{biz}
If $L$ is a regular language, then  there is such an DFA which accepts $L$  (by Theorems \ref{KZregnemdet} and \ref{KZnemdetdet}). Let ${\mathrm A}=(Q,\Sigma, E, \{q_0\},F)$ be  this DFA, so $L=L({\mathrm A})$. Let $n$ be 
the number of its states, that is $|Q|=n$. Let $u=a_1a_2\ldots a_m\in L$ and  $m\ge n$.
Then, because the automaton accepts word  $u$, there are states  $q_0,q_1,\ldots, q_m$
and walk
\[ q_0 \overset{a_1}{\longrightarrow} q_1 \overset{a_2}{\longrightarrow} q_2 \overset{a_3}{\longrightarrow} \;\cdots 
\; \overset{a_{m-1}}{\longrightarrow} q_{m-1} \overset{a_m}{\longrightarrow} q_m, \; q_m\in F \koz .\]
\begin{figure}\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(95.00,49.00)
\put(8.00,6.00){\circle{8.00}}
\put(19.00,17.00){\circle{8.00}}
\put(30.00,21.00){\circle{8.00}}
\put(58.00,21.00){\circle{8.00}}
\put(70.00,30.00){\circle{8.00}}
\put(70.00,45.00){\circle{8.00}}
\put(50.00,33.00){\circle{8.00}}
\put(68.00,10.00){\circle{8.00}}
\put(91.00,10.00){\circle{8.00}}
\put(8.00,6.00){\makebox(0,0)[cc]{{\small $q_0$}}}
\put(19.00,17.00){\makebox(0,0)[cc]{{\small $q_1$}}}
\put(30.00,21.00){\makebox(0,0)[cc]{{\small $q_2$}}}
\put(58.00,21.00){\makebox(0,0)[cc]{\small {$q_j$}}}
\put(68.00,10.00){\makebox(0,0)[cc]{{\small $q_{k+1}$}}}
\put(91.00,10.00){\makebox(0,0)[cc]{{\small $q_m$}}}
\put(60.00,13.00){\makebox(0,0)[rc]{{\small $q_j=q_k$}}}
\put(15.00,10.00){\makebox(0,0)[cc]{{\small $a_1$}}}
\put(26.00,16.00){\makebox(0,0)[cc]{{\small $a_2$}}}
\put(37.00,18.00){\makebox(0,0)[cc]{{\small $a_3$}}}
\put(85.00,7.00){\makebox(0,0)[cc]{{\small $a_m$}}}
\put(67.00,24.00){\makebox(0,0)[cc]{{\small $a_{j+1}$}}}
\put(52.00,26.00){\makebox(0,0)[cc]{{\small $a_k$}}}
\put(51.00,19.00){\makebox(0,0)[cc]{{\small $a_j$}}}
\put(45.00,21.00){\makebox(0,0)[cc]{$\ldots $}}
\put(79.00,10.00){\makebox(0,0)[cc]{$\ldots $}}
\put(1.00,6.00){\vector(1,0){3.00}}
\put(91.00,10.00){\circle{6.00}}
\put(9.80,9.80){\vector(3,2){6.40}}
\put(34.00,21.00){\vector(1,0){6.00}}
\put(49.00,21.00){\vector(1,0){5.00}}
\put(72.00,10.00){\vector(1,0){3.00}}
\put(82.00,10.00){\vector(1,0){5.00}}
\put(61.00,24.00){\vector(4,3){5.50}}
\put(70.00,34.00){\vector(0,1){7.00}}
\put(64.00,16.00){\makebox(0,0)[lc]{$a_{k+1}$}}
\put(67.00,42.00){\vector(-4,-3){4.00}}
\put(58.00,36.00){\vector(-4,-3){4.00}}
\put(62.00,38.00){\makebox(0,0)[cc]{.}}
\put(61.00,37.50){\makebox(0,0)[cc]{.}}
\put(60.00,37.00){\makebox(0,0)[cc]{.}}
\put(23.10,17.90){\vector(3,2){3.40}}
\put(52.20,29.80){\vector(3,-4){4.00}}
\put(61.00,18.00){\vector(3,-4){4.00}}
\put(50.00,33.00){\makebox(0,0)[cc]{${\small q_{k-1}}$}}
\put(70.00,30.00){\makebox(0,0)[cc]{${\small q_{j+1}}$}}
\end{picture}
\caption{Sketch of DFA used in the proof of the pumping lemma.}
\label{KZaut1}
\end{figure}
Because the number of states is  $n$ and $m\ge n$, by the pigeonhole 
principle\footnote{ \emph{Pigeonhole principle:} 
If we have to put more than $k$ objects into  $k$ boxes, then at least one box will contain at least two objects.} 
states  $q_0, q_1,\ldots, q_m$  can not all be distinct (see Fig. \ref{KZaut1}), 
there are at least two of them  which are equal. Let $q_j=q_k$, where $j<k$ and $k$ is the least such index. 
Then $j<k\le n$. Decompose word  $u$ as: 

$x=a_1a_2\ldots a_j$

$y=a_{j+1}a_{j+2}\ldots a_k$

$z=a_{k+1}a_{k+2}\ldots a_m$ \koz .

\noindent This decomposition immediately yields to  $|xy|\le n$ and $|y|\ge 1$. We will prove that $xy^iz\in L$ for any   $i$.

\noindent Because $u=xyz\in L$, there exists an walk  
\[ q_0 \overset{x}{\longrightarrow} q_j \overset{y}{\longrightarrow} q_k \overset{z}{\longrightarrow} q_m, \; q_m\in F,\] and because of $q_j=q_k$, this may be written also as
\[ q_0 \overset{x}{\longrightarrow} q_j \overset{y}{\longrightarrow} q_j 
\overset{z}{\longrightarrow} q_m, \; q_m\in F \koz .\]  
From this walk $q_j\overset{y}{\longrightarrow} q_j$  can be omitted or can be inserted many times.  So, there are the following walks: 
\[ q_0 \overset{x}{\longrightarrow} q_j  \overset{z}{\longrightarrow} q_m, \; q_m\in F \koz ,\]  
\[ q_0 \overset{x}{\longrightarrow} q_j \overset{y}{\longrightarrow} q_j \overset{y}{\longrightarrow} 
\ldots \overset{y}{\longrightarrow} q_j \overset{z}{\longrightarrow} q_m, \; q_m\in F \koz .\]  
Therefore $xy^iz\in L$ for all $i$, and this proves the theorem.
\end{biz}

\begin{pelda} We use the pumping lemma  to show that $L_1=\{a^kb^k\;|\; k\ge 1\}$ is not regular.
Assume that $L_1$ is regular, and let $n$ be the corresponding natural number  given by the pumping lemma.  Because the length of the word $u=a^nb^n$ is  $2n$, this word can be written as in the lemma. We prove that this leads to a contradiction. Let $u=xyz$ be the decomposition as in the lemma. Then  $|xy|\le n$, so $x$ and $y$ can contain no other  letters than $a$, and because we must have $|y|\ge 1$, word $y$ contains at least one $a$.
Then $xy^iz$ for $i\not= 1$ will contain a different number of  $a$'s and  $b$'s, therefore  $xy^iz\not\in L_1$ for any  $i\not = 1$. This is a contradiction with the third assertion of the lemma, this is why that assumption that $L_1$ is regular, is false.  Therefore $L_1\not\in {\mathcal L}_3$.

Because the context-free grammar $G_1=(\{S\}, \{a,b\}, \{S\rightarrow ab, S\rightarrow aSb\}, S)$ generates language $L_1$, we have $L_1\in {\mathcal L}_2$. 
From these two follow that ${\mathcal L}_3 \subset {\mathcal L}_2.$
\end{pelda}
\begin{pelda} We show that  $L_2=\big\{u\in \{0,1\}^*\;|\; n_0(u)=n_1(u)\big\}$ is not regular. ($n_0(u)$ is the number of 0's in $u$, while  $n_1(u)$ the number of 1's).

We proceed as in the previous example using here word $u=0^n1^n$, where $n$ is the natural number associated by lemma to language $L_2$.  
\end{pelda}
\begin{pelda} We prove, using the pumping lemma, that  $L_3=\big\{uu \;|\; u\in \{a,b\}^*\big\}$ is not a regular language. Let $w=a^nba^nb=xyz$ be, where $n$ here is also the natural number associated to  $L_3$ by the pumping lemma. From 
$|xy|\le n$ we have that $y$ contains no other letters than $a$, but it contains at least one. By lemma we have $xz\in L_3$, that is not possible.  Therefore $L_3$ is not regular.
\end{pelda}

Pumping lemma has several interesting consequences.

\begin{kov} \label{KZkov16}
Regular language $L$ is not empty if and only if there exists a word 
$u\in L$, $|u|<n$, where  $n$ is the natural number associated to $L$ by the pumping lemma. \end{kov}
\begin{biz}
The assertion in a direction is obvious: if there exists a word shorter than $n$ in 
$L$, then $L \not= \emptyset$. Conversely, let $L\not=\emptyset$ and  
let $u$ be the shortest word in  $L$. We show that $|u|<n$. If $|u|\ge n$, then we apply the pumping lemma, and give the decomposition  $u=xyz$, $|y|>1$ and $xz\in L$. This is a contradiction, because  $|xz|<|u|$ and  $u$ is the shortest word in $L$. Therefore $|u|<n$.  \end{biz}
\begin{kov}
There exists an algorithm that can decide if a regular language is or not empty.
\end{kov}
\begin{biz} Assume that  $L=L({\mathrm A})$, where ${\mathrm A}=(Q,\Sigma, E, \{q_0\},F)$ is a DFA. 
By Consequence \ref{KZkov16} and Theorem  \ref{KZiteracios} language $L$ is not empty if and only if it 
contains a word shorter than $n$, where $n$ is the number of states of automaton ${\mathrm A}$. By this it is 
enough to decide that there is a word shorter than $n$ which is accepted by automaton ${\mathrm A}$. 
Because the number of words shorter than  $n$ is finite, the problem can be decided.
\end{biz}

When we had given an algorithm for inaccessible states of a DFA, we remarked that the procedure can be used also to decide 
if the language accepted by that automaton is or not empty. Because finite automata accept regular languages, 
we can consider to have already two 
procedures to decide if a regular languages is or not empty. Moreover, we have a third procedure, 
if we take into account that the algorithm for finding productive states also can be used to decide on a regular 
language when it is empty.
\begin{kov}
A regular language $L$ is infinite if and only if 
there exists a word $u\in L$ such that $n\le |u|<2n$, where $n$ is the natural number associated to language $L$, 
given by the pumping lemma.
\end{kov} 
\begin{biz}
If $L$ is infinite, then it contains words longer than $2n$, and let  $u$ be 
the shortest word longer than  $2n$  in $L$.
Because  $L$ is regular we can use the pumping lemma, so 
$u=xyz$, where $|xy|\le n$, thus  $|y|\le n$ is also true. By the lemma  
$u'=xz\in L$. But because $|u'|<|u|$ and the shortest word in $L$ longer than  $2n$ is $u$, we get  $|u'|< 2n$. From  $|y|\le n$ we get also $|u'|\ge n$.

Conversely, if there exists a word  $u\in L$ such that $n\le |u|<2n $, then using the pumping lemma, we obtain that $u=xyz$, $|y|\ge 1$  and  $xy^iz\in L$ for any  
$i$, therefore  $L$ is infinite. 
\end{biz}


Now, the question is: how can we apply the pumping lemma for a finite regular language, since by pumping words we get an infinite number of words? The number of states of a DFA accepting language $L$ is greater than the length of the longest word in $L$. So, in $L$ there is no word with length at least $n$, when $n$ is the natural number associated to $L$ by the pumping lemma. Therefore, no word in $L$ can be decomposed in the form  $xyz$, where 
$|xyz|\ge n$, $|xy|\le n$, $|y|\ge 1$,  and this is why we can not obtain an infinite number of words in $L$.

\subsection{Regular expressions}
In this subsection we introduce for any alphabet $\Sigma$  the notion of regular expressions over $\Sigma$ and the corresponding representing languages. A regular expression   \inddef{regular expression} is a formula,  and the corresponding language is a language over $\Sigma$. For example, if  $\Sigma=\{a,b\}$, then $a^*$, $b^*$, $a^*+b^*$ are regular expressions over $\Sigma$ which represent respectively languages $\{a\}^*$, $\{b\}^*$, $\{a\}^*\cup \{b\}^*$.
The exact definition is the following.
\vspace{-2mm}
\begin{defi}
Define recursively a regular expression over $\Sigma$ and the language it represent.

$\bullet$ $\emptyset$  is a regular expression representing the empty language.

$\bullet$  $\varepsilon$  is a regular expression representing language  $\{\varepsilon\}$.

$\bullet$  If $a\in \Sigma$,  then $a$ is a regular expression representing  language  $\{a\}$.

$\bullet$  If  $x$, $y$ are regular expressions representing languages $X$ and $Y$ respectively, 
then $(x+y)$, $(xy)$, $(x^*)$ are regular expressions 
representing  languages  $X\cup Y$, $XY$ and $X^*$ respectively. 

Regular expression over  $\Sigma$ can be obtained only by using the above rules a finite  number of times. 
\end{defi}
Some brackets can be omitted in the regular expressions if taking into account the priority 
of operations (iteration, product, union) the corresponding languages are not affected.
For example instead of $((x^*)(x+y))$ we can consider $x^*(x+y)$.   

Two regular expressions are  \ki{equivalent}\inddef{equivalent expressions} 
if they represent the same language, that is $x\equiv y$ if  $X=Y$, where 
$X$ and $Y$ are the languages represented by regular expressions  $x$ and $y$ respectively. 
Figure \ref{KZregkifej} shows some equivalent expressions.
\begin{figure}
\begin{eqnarray*}
x+y&\equiv &y+x\\
(x+y)+z&\equiv &x+(y+z)\\
(xy)z&\equiv &x(yz)\\
(x+y)z&\equiv &xz+yz\\
x(y+z)&\equiv &xy+xz\\
(x+y)^*\;\equiv \;(x^*+y)^*&\equiv &(x+y^*)^*\;\equiv \;(x^*+y^*)^* \\
(x+y)^*&\equiv &(x^*y^*)^*\\
(x^*)^*&\equiv &x^*\\
x^*x&\equiv &xx^*\\
xx^*+\varepsilon&\equiv & x^*
\end{eqnarray*}
\caption{Properties of regular expressions.}
\label{KZregkifej}
\end{figure}

We show that to any finite language  $L$ can be associated a regular expression $x$ which represent language  $L$. If $L=\emptyset$, then $x=\emptyset$. If $L= \{w_1, w_2, \ldots, w_n\}$, then $x=x_1+x_2+ \ldots+ x_n$, where for any   
$i=1,2,\ldots, n $ expression  $x_i$ is a regular expression representing language  $\{w_i\}$. This latter can be done by the following rule. If  $w_i=\varepsilon$, then $x_i=\varepsilon$, else if $ w_i=a_1a_2\ldots a_m$, where $m\ge 1$ depends on $i$,  then  $x_i=a_1a_2\ldots a_m$, where the brackets are omitted. 
 
We prove the theorem of Kleene which refers to the relationship between regular languages and regular expression.\nevindex{Kleene, Stephen C.}
 
\begin{tetel}[Kleene's theorem] Language $L\subseteq \Sigma^*$ is regular if and only if  
\index{Kleene's theorem}\index{regular!expression} there exists a regular expression over $\Sigma$ representing language $L$.
\end{tetel}
\begin{biz}
First we prove that if  $x$ is a regular expression, then language $L$ which represents $x$ is also regular. 
The proof will be done by induction on the construction of expression. 

If $x=\emptyset$, $x=\varepsilon$, $x=a, \forall a\in \Sigma$, then $L=\emptyset$, $L=\{\varepsilon\}$, 
$L=\{a\}$ respectively. Since $L$ is finite in all three cases, it is also regular. 

If $x=(x_1+x_2)$, then $L=L_1\cup L_2$, where $L_1$ and $L_2$ are the languages which represent 
the regular expressions  $x_1$ and  $x_2$ respectively.  By the induction  hypothesis languages $L_1$ and $L_2$ 
are regular, so $L$ is also regular  because regular languages are closed on union.  
Cases $x=(x_1x_2)$ and $x=(x_1^*)$ can be proved by similar way.



Conversely, we prove that if $L$ is a regular language, then a regular expression $x$ can be associated to it, which represent exactly the language $L$. If $L$ is regular, then there exists a DFA  ${\mathrm A}=(Q,\Sigma,E,\{q_0\},F)$ for which $L=L({\mathrm A})$.
Let  $q_0, q_1, \ldots, q_n$ the states of the automaton ${\mathrm A}$. Define languages $R_{ij}^k$ for all $-1\le k\le n$  
and $0\le i,j\le n$.
$R_{ij}^k$ is the set of words, for which automaton ${\mathrm A}$ goes from state $q_i$ 
to state  $q_j$ without using any state with index greater  than $k$. Using transition graph we can say: a word is in $R_{ij}^k$, if  from state $q_i$ we arrive to state $q_j$ following the edges of the graph, and concatenating the corresponding labels on edges we get exactly that word,  not using any state  $q_{k+1}, \ldots q_n$.  Sets $R_{ij}^k$ can be done also formally:

$R_{ij}^{-1}= \{a\in \Sigma\;|\; (q_i,a,q_j)\in E\}$, if $i\not = j$,

$R_{ii}^{-1}= \{a\in \Sigma\;|\; (q_i,a,q_i)\in E\}\cup \{\varepsilon\}$,

\smallskip
$R_{ij}^k= R_{ij}^{k-1}\cup R_{ik}^{k-1}\left(R_{kk}^{k-1} \right)^*R_{kj}^{k-1}$ 
for all  $i,j,k\in\{0,1,\ldots, n\}$.

We can prove by induction that sets $R_{ij}^k$  can be described by regular expressions. 
 Indeed, if $k=-1$, then for all $i$ and $j$ languages  $R_{ij}^k$ are finite, so they can be expressed 
 by regular expressions representing exactly these languages. Moreover, if for all $i$ and  $j$ language  
 $R_{ij}^{k-1}$ can be expressed by regular expression, then 
language $R_{ij}^k$ can be expressed also by regular expression, which can be corresponding 
constructed from regular expressions representing languages $R_{ij}^{k-1}$, $R_{ik}^{k-1}$, $R_{kk}^{k-1}$ 
and $R_{kj}^{k-1}$ respectively,  using the above formula for $R_{ij}^k$.   

Finally, if $F=\{q_{i_1}, q_{i_2}, \ldots, q_{i_p}\}$  is the set of final states of the DFA ${\mathrm A}$, 
then  $ L=L({\mathrm A})= R_{0i_1}^n\cup R_{0i_2}^n\cup \ldots \cup R_{0i_p}^n$
can be expressed by a regular expression obtained from  expressions representing languages   
$R_{0i_1}^n, R_{0i_2}^n, \ldots , R_{0i_p}^n$ using operation  $+$. 
\end{biz}

Further on we give some procedures which associate DFA to regular expressions and conversely regular expression to DFA.  

\subsubsection{Associating  regular expressions to finite automata.}
We present here three methods, each of which associate to a DFA the corresponding regular expression. 

\textit{Method 1.} Using the result of the theorem of Kleene, we will construct the sets $R^k_{ij}$, 
and write a regular expression which represents the language $L=R_{0i_1}^n\cup R_{0i_2}^n\cup \ldots \cup R_{0i_p}^n$, where $F=\{q_{i_1}, q_{i_2}, \ldots, q_{i_p}\}$ is the set of final states of the automaton.
\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(55.00,24.00)
\put(10.00,11.00){\circle{10.00}}
\put(50.00,11.00){\circle{10.00}}
\put(10.00,11.00){\circle{8.00}}
\put(10.00,11.00){\makebox(0,0)[cc]{$q_0$}}
\put(50.00,11.00){\makebox(0,0)[cc]{$q_1$}}
\put(1.00,11.00){\vector(1,0){4.00}}
\bezier{152}(13.00,15.00)(30.00,24.00)(47.00,15.00)
\bezier{148}(14.00,7.00)(30.00,-1.00)(47.00,7.00)
\put(45.70,6.30){\vector(2,1){1.00}}
\put(14.80,15.80){\vector(-2,-1){1.00}}
\put(50.00,15.00){\oval(6.00,12.00)[t]}
\put(10.00,15.00){\oval(6.00,12.00)[t]}
\put(10.00,23.00){\makebox(0,0)[cc]{$1$}}
\put(50.00,23.00){\makebox(0,0)[cc]{$0$}}
\put(31.00,22.00){\makebox(0,0)[cc]{1}}
\put(31.00,5.00){\makebox(0,0)[cc]{0}}
\put(7.00,17.00){\vector(0,-1){2.00}}
\put(53.00,17.00){\vector(0,-1){2.00}}
\end{picture}
\caption{DFA from Example \ref{KZregkiff}, to which regular expression is associated by Method 1.}
\label{KZ4ka1}
\end{figure}


\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(90.00,35.00)
\put(10.00,17.00){\circle{10.00}}
\put(35.00,17.00){\circle{10.00}}
\put(60.00,17.00){\circle{10.00}}
\put(85.00,17.00){\circle{10.00}}
\put(85.00,17.00){\circle{8.00}}
\put(10.00,17.00){\makebox(0,0)[cc]{$q_0$}}
\put(35.00,17.00){\makebox(0,0)[cc]{$q_1$}}
\put(60.00,17.00){\makebox(0,0)[cc]{$q_2$}}
\put(85.00,17.00){\makebox(0,0)[cc]{$q_3$}}
\put(1.00,17.00){\vector(1,0){4.00}}
\put(15.00,17.00){\vector(1,0){15.00}}
\put(40.00,17.00){\vector(1,0){15.00}}
\put(65.00,17.00){\vector(1,0){15.00}}
\put(60.50,21.50){\oval(7.00,15.00)[t]}
\put(64.00,23.00){\vector(0,-1){3.00}}
\bezier{212}(15.00,19.00)(35.00,35.00)(55.00,19.00)
\bezier{208}(38.00,13.00)(60.00,-2.00)(82.00,13.00)
\put(54.30,19.80){\vector(1,-1){1.00}}
\put(80.80,12.00){\vector(1,1){1.00}}
\put(21.00,14.00){\makebox(0,0)[cc]{1}}
\put(47.00,14.00){\makebox(0,0)[cc]{0}}
\put(72.00,14.00){\makebox(0,0)[cc]{1}}
\put(61.00,8.00){\makebox(0,0)[cc]{1}}
\put(35.00,29.00){\makebox(0,0)[cc]{0}}
\put(61.00,31.00){\makebox(0,0)[cc]{0}}
\end{picture}
\caption{DFA in Example \ref{KZautreg2} to which a regular expression is associated by Method 1. 
The computation are in Figure \ref{KZtabla1}.}
\label{KZ4ka2}
\end{figure}

\begin{figure}[h!]\centering
\begin{tabular}{|c||c|c|c|c|c|}\hline
           &$k=-1$& $k=0$  &  $ k=1$   & $k=2$   & $k=3$ \\ \hline\hline
           &      &        &           &         &        \\
$ R^k_{00}$&$\varepsilon$& $\varepsilon$&$\varepsilon $ &  $\varepsilon $    &    \\
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{01}$& $\qquad$ 1$\qquad$  & $\qquad$  1 $\qquad$  & $\qquad$ 1 $\qquad$     &  $1 $    &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{02}$& 0           & $0 $   &  $0+10 $  &  $(0+10)0^* $ &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{03}$&$\emptyset$  & $\emptyset$    &  $11$     &  $11+(0+10)0^*1$  & $11+(0+10)0^*1 $ \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{11}$&$\varepsilon$&$\varepsilon$ &$\varepsilon$  &  $\varepsilon$ &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{12}$&0            & $0 $   &  $0 $  &  $00^* $ &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{13}$&1            & $1 $   &  $1$      &  $1+00^*1 $ &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{22}$&$0+\varepsilon$ & $0+\varepsilon$ &  $0+\varepsilon  $   &  $0^*$  &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{23}$&1            & $1 $   &  $1$      &  $0^*1$ &  \\ 
           &             &        &           &         &        \\ \hline
           &             &        &           &         &        \\
$ R^k_{33}$&$\varepsilon$&$\varepsilon$&$\varepsilon$ &$\varepsilon$&  \\ 
           &             &        &           &         &        \\ \hline
\end{tabular}\caption{Determining a regular expression associated to DFA in Figure \ref{KZ4ka2} using sets  $R^k_{ij}$.}
\label{KZtabla1}
\end{figure}

\begin{pelda}\label{KZregkiff}
Consider the DFA in Fig. \ref{KZ4ka1}.
 
$L({\mathrm A})=R_{00}^1= R_{00}^0\cup R_{01}^0\left( R_{11}^0\right)^*R_{10}^0$

$R_{00}^0:$ $\quad 1^*+\varepsilon\equiv 1^*$

$R_{01}^0:$ $\quad 1^*0$

$R_{11}^0:$ $\quad 11^*0+\varepsilon + 0 \equiv (11^*+\varepsilon)0+\varepsilon \equiv 1^*0+\varepsilon$

$R_{10}^0:$ $\quad 11^*$

Then the regular expression corresponding to  $L({\mathrm A})$ is $1^*+1^*0(1^*0+\varepsilon)^*11^* \equiv 1^* + 1^*0(1^*0)^*11^*$.
\end{pelda}

\begin{pelda} \label{KZautreg2} Find a regular expression associated to DFA in Fig. \ref{KZ4ka2}. The computations 
are in Figure  \ref{KZtabla1}. The regular expression corresponding to $R^3_{03}$ is $11+(0+10)0^*1$.
\end{pelda}


\textit{Method 2.} Now we generalize the notion of finite automaton, considering words instead of letters as labels of
edges. In such an automaton each walk determine a regular expression, which determine a regular language.  
The regular language accepted by a generalized finite automaton is the union of regular languages determined by the 
productive walks. It is easy to see that the generalized finite automata accept regular languages.

The advantage of generalized finite automata is that the number of its edges can be diminuted  by equivalent transformations, which do not change the accepted language, and leads to a graph with only one edge which label is exactly the accepted language.
    
The possible equivalent transformations can be seen in Fig.  \ref{KZvegreg}. If some of the vertices 1, 2, 4, 5 on the figure coincide, in the result they are merged, and a loop will arrive.

First, the automaton is transformed by corresponding  $\varepsilon$-moves to have only one initial and one final state. 
Then, applying the equivalent transformations until the graph will have only one edge, we will obtain as the label of this edge the regular expression associated to the automaton.

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(110.00,21.00)
\put(5.00,10.00){\circle{6.00}}
\put(28.00,10.00){\circle{6.00}}
\bezier{112}(5.00,13.00)(17.00,21.00)(28.00,13.00)
\bezier{112}(5.00,7.00)(17.00,-1.00)(28.00,7.00)
\put(27.00,14.00){\vector(1,-1){1.00}}
\put(27.00,6.00){\vector(1,1){1.00}}
\put(17.00,20.00){\makebox(0,0)[cc]{$x$}}
\put(17.00,5.00){\makebox(0,0)[cc]{$y$}}
\put(85.00,10.00){\circle{6.00}}
\put(107.00,10.00){\circle{6.00}}
\put(88.00,10.00){\vector(1,0){16.00}}
\put(96.00,13.00){\makebox(0,0)[cc]{$x+y$}}
\put(57.00,10.00){\makebox(0,0)[cc]{replaced by}}
\end{picture}

\medskip
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(112.00,31.00)
\put(5.00,5.00){\circle{6.00}}
\put(40.00,5.00){\circle{6.00}}
\put(23.00,17.00){\circle{6.00}}
\put(5.00,28.00){\circle{6.00}}
\put(40.00,28.00){\circle{6.00}}
\put(74.00,5.00){\circle{6.00}}
\put(109.00,5.00){\circle{6.00}}
\put(74.00,28.00){\circle{6.00}}
\put(109.00,28.00){\circle{6.00}}
\put(8.00,27.00){\vector(3,-2){12.80}}
\put(25.00,19.00){\vector(3,2){12.00}}
\put(8.00,6.00){\vector(3,2){13.00}}
\put(25.00,15.00){\vector(3,-2){12.80}}
\put(22.50,20.00){\oval(5.00,8.00)[t]}
\put(25.00,21.00){\vector(0,-1){1.00}}
\put(14.00,25.00){\makebox(0,0)[cc]{$x$}}
\put(23.00,26.00){\makebox(0,0)[cc]{$y$}}
\put(30.00,25.00){\makebox(0,0)[cc]{$z$}}
\put(15.00,8.00){\makebox(0,0)[cc]{$u$}}
\put(30.00,9.00){\makebox(0,0)[cc]{$v$}}
\put(77.00,28.00){\vector(1,0){29.00}}
\put(77.00,5.00){\vector(1,0){29.00}}
\put(77.00,28.00){\vector(3,-2){30.80}}
\put(77.00,5.00){\vector(3,2){30.40}}
\put(57.00,16.00){\makebox(0,0)[cc]{replaced by}}
\put(92.00,31.00){\makebox(0,0)[cc]{$xy^*z$}}
\put(92.00,2.00){\makebox(0,0)[cc]{$uy^*v$}}
\put(103.00,20.00){\makebox(0,0)[lc]{$uy^*z$}}
\put(103.00,13.00){\makebox(0,0)[lc]{$xy^*v$}}
\put(5.00,28.00){\makebox(0,0)[cc]{{\small 1}}}
\put(40.00,28.00){\makebox(0,0)[cc]{{\small 2}}}
\put(23.00,17.00){\makebox(0,0)[cc]{{\small 3}}}
\put(5.00,5.00){\makebox(0,0)[cc]{{\small 4}}}
\put(40.00,5.00){\makebox(0,0)[cc]{{\small 5}}}
\put(74.00,28.00){\makebox(0,0)[cc]{{\small 1}}}
\put(109.00,28.00){\makebox(0,0)[cc]{{\small 2}}}
\put(74.00,5.00){\makebox(0,0)[cc]{{\small 4}}}
\put(109.00,5.00){\makebox(0,0)[cc]{{\small 5}}}
\end{picture}
\caption{Possible equivalent transformations for finding regular expression associated to an automaton.}
\label{KZvegreg}
\end{figure}


\begin{figure}[t!]
\hspace*{-0.5cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(48.00,59.00)
\put(18.00,29.00){\circle{10.00}}
\put(43.00,29.00){\circle{10.00}}
\put(18.00,29.00){\makebox(0,0)[cc]{$q_0$}}
\put(43.00,29.00){\makebox(0,0)[cc]{$q_1$}}
\put(38.60,24.60){\vector(1,1){1.00}}
\put(22.60,33.60){\vector(-1,-1){1.00}}
\put(43.00,33.00){\oval(6.00,12.00)[t]}
\put(43.00,41.00){\makebox(0,0)[cc]{$0$}}
\put(31.00,40.00){\makebox(0,0)[cc]{1}}
\put(31.00,24.00){\makebox(0,0)[cc]{0}}
\put(46.00,35.00){\vector(0,-1){2.00}}
\put(18.00,29.00){\circle{10.00}}
\put(18.00,54.00){\circle{10.00}}
\put(18.00,7.00){\circle{10.00}}
\put(10.00,54.00){\vector(1,0){3.00}}
\put(18.00,24.00){\vector(0,-1){12.00}}
\put(18.00,7.00){\circle{8.00}}
\put(14.50,29.00){\oval(16.00,8.00)[l]}
\put(12.00,33.00){\vector(1,0){3.00}}
\put(3.00,29.00){\makebox(0,0)[cc]{1}}
\put(18.00,49.00){\vector(0,-1){15.00}}
\put(21.00,43.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(21.00,18.00){\makebox(0,0)[cc]{$\varepsilon$}}
\bezier{92}(22.00,33.00)(32.00,40.00)(40.00,33.00)
\bezier{92}(39.00,25.00)(31.00,18.00)(21.00,25.00)
\end{picture}
\qquad
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(32.00,59.00)
\put(17.00,29.00){\circle{10.00}}
\put(17.00,29.00){\makebox(0,0)[cc]{$q_0$}}
\put(17.00,29.00){\circle{10.00}}
\put(17.00,54.00){\circle{10.00}}
\put(17.00,7.00){\circle{10.00}}
\put(9.00,54.00){\vector(1,0){3.00}}
\put(17.00,24.00){\vector(0,-1){12.00}}
\put(17.00,7.00){\circle{8.00}}
\put(13.40,29.00){\oval(18.00,8.00)[l]}
\put(11.00,33.00){\vector(1,0){3.00}}
\put(2.00,29.00){\makebox(0,0)[cc]{1}}
\put(17.00,49.00){\vector(0,-1){15.00}}
\put(20.30,29.00){\oval(18.00,8.00)[r]}
\put(22.00,33.00){\vector(-1,0){2.00}}
\put(32.00,29.00){\makebox(0,0)[lc]{$00^*1$}}
\put(20.00,41.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(20.00,17.00){\makebox(0,0)[cc]{$\varepsilon$}}
\end{picture}
\qquad
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(14.00,34.00)
\put(9.00,29.00){\circle{10.00}}
\put(9.00,7.00){\circle{10.00}}
\put(1.00,29.00){\vector(1,0){3.00}}
\put(9.00,7.00){\circle{8.00}}
\put(9.00,24.00){\vector(0,-1){12.00}}
\put(12.00,18.00){\makebox(0,0)[lc]{$(1+00^*1)^*$}}
\end{picture}
\caption{Transformation of the finite automaton in Fig.  \ref{KZ4ka1}.}
\label{KZ4kx}
\end{figure}

\begin{pelda}  In the case of Fig.  \ref{KZ4ka1} the result is obtained by steps illustrated in Fig. \ref{KZ4kx}. 
 This result is  $(1+00^*1)^*$, which represents the same language as obtained by Method 1 (See example \ref{KZregkiff}).
\end{pelda}

\begin{pelda}\label{KZ4ka}
In the case of Fig. \ref{KZ4ka2} is not necessary to introduce new initial and final state. 
 The steps of transformations can be seen in Fig. \ref{KZ4kz}.  The resulted regular expression can be written also as  
 $(0+10)0^*1+11$, which is the same as obtained by the previous method. 
\end{pelda}

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(90.00,37.00)
\put(10.00,19.00){\circle{10.00}}
\put(60.00,19.00){\circle{10.00}}
\put(85.00,19.00){\circle{10.00}}
\put(85.00,19.00){\circle{8.00}}
\put(10.00,19.00){\makebox(0,0)[cc]{$q_0$}}
\put(60.00,19.00){\makebox(0,0)[cc]{$q_2$}}
\put(85.00,19.00){\makebox(0,0)[cc]{$q_3$}}
\put(1.00,19.00){\vector(1,0){4.00}}
\put(65.00,19.00){\vector(1,0){15.00}}
\put(60.00,23.00){\oval(7.00,15.00)[t]}
\put(63.50,25.40){\vector(0,-1){3.00}}
\bezier{212}(14.50,21.00)(35.00,37.00)(55.50,21.00)
\put(54.50,22.10){\vector(1,-1){1.00}}
\put(72.00,16.00){\makebox(0,0)[cc]{1}}
\put(35.00,31.00){\makebox(0,0)[cc]{0}}
\put(61.00,32.00){\makebox(0,0)[cc]{0}}
\put(15.00,18.00){\vector(1,0){40.00}}
\put(35.00,20.00){\makebox(0,0)[cc]{$10$}}
\bezier{312}(13.50,15.00)(45.00,-3.00)(82.00,15.00)
\put(81.00,14.30){\vector(2,1){1.00}}
\put(46.00,2.00){\makebox(0,0)[cc]{$11$}}
\end{picture}

\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(90.00,43.00)
\put(10.00,20.00){\circle{10.00}}
\put(85.00,20.00){\circle{10.00}}
\put(85.00,20.00){\circle{8.00}}
\put(10.00,20.00){\makebox(0,0)[cc]{$q_0$}}
\put(85.00,20.00){\makebox(0,0)[cc]{$q_3$}}
\put(1.00,20.00){\vector(1,0){4.00}}
\bezier{312}(13.40,16.00)(45.00,-2.00)(82.00,16.00)
\put(81.00,15.30){\vector(2,1){1.00}}
\put(48.00,3.00){\makebox(0,0)[cc]{$11$}}
\bezier{316}(13.00,24.00)(46.00,43.00)(83.00,25.00)
\put(81.00,26.00){\vector(2,-1){2.00}}
\put(15.00,19.00){\vector(1,0){65.00}}
\put(48.00,38.00){\makebox(0,0)[cc]{$00^*1$}}
\put(48.00,22.00){\makebox(0,0)[cc]{$100^*1$}}
\end{picture}

\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(90.00,11.00)
\put(10.00,6.00){\circle{10.00}}
\put(85.00,6.00){\circle{10.00}}
\put(85.00,6.00){\circle{8.00}}
\put(10.00,6.00){\makebox(0,0)[cc]{$q_0$}}
\put(85.00,6.00){\makebox(0,0)[cc]{$q_3$}}
\put(1.00,6.00){\vector(1,0){4.00}}
\put(48.00,8.00){\makebox(0,0)[cc]{$00^*1+100^*1+11$}}
\put(15.00,6.00){\vector(1,0){65.00}}
\end{picture}
\caption{Steps of Example  \ref{KZ4ka}.}
\label{KZ4kz}
\end{figure}

\textit{Method 3.} The third method for writing regular expressions associated to finite automata uses formal equations. A variable $X$ is associated  to each state of the automaton (to different states different variables).  
Associate to each state an equation which left side contains $X$, its right side contains sum of terms of form 
  $Ya$ or $\varepsilon$, where $Y$ is a variable associated to a state, and $a$ is its corresponding input symbol. If there is no incoming edge in the state corresponding to $X$ then the right side of the equation with left side $X$ contains $\varepsilon$, otherwise is the sum of all terms of the form  $Ya$ for which there is  a transition labelled with letter $a$ from state corresponding to  $Y$ to the state corresponding to  $X$. If the state corresponding to 
 $X$ is also an initial and a final state, then on right side of the equation with the left side $X$ will be also a term equal to $\varepsilon$. For example in the case of Fig.  \ref{KZ4ka2} let these variable  $X,Y,Z,U$ corresponding to the states $q_0,q_1,q_2,q_3$. The corresponding equation are

$X=\varepsilon$

$Y=X1$

$Z=X0+Y0+Z0$

$U=Y1+Z1 .$

If an equation is of the form  $X=X\alpha+\beta$, where $\alpha, \beta$  are arbitrary words not containing $X$, then  it is easy to see by a simple substitution that $X=\beta\alpha^*$ is a solution of the equation. 

Because these equations are linear, all of them can be written in the form $X=X\alpha+\beta$ or  $X=X\alpha$, where $\alpha$ do not contain any variable. Substituting this in the other equations the number of remaining equations will be diminuted by one. In such a way the system of equation can be solved for each variable.

The solution will be given by variables corresponding to final states summing the corresponding regular expressions. 

In our example from the first equation we get  $Y=1.$
From here  $\quad Z=0+10+Z0, \; \textrm{ or }\; Z=Z0+(0+10),$
and solving this we get  $Z=(0+10)0^*.$
Variable $U$ can be obtained immediately and we obtain  $U=11+(0+10)0^*1.$

Using this method in the case of Fig. \ref{KZ4ka1}, the following equations will be obtained

$X=\varepsilon +X1+Y1$

$Y=X0+Y0$

\noindent Therefore 

$X= \varepsilon +(X+Y)1$

$Y=(X+Y)0.$

\noindent 
Adding the two equations we will obtain

$X+Y= \varepsilon +(X+Y)(0+1)$, from where (considering $\varepsilon$ as $\beta$ and  
$(0+1)$ as $\alpha$) we get the result 

$X+Y= (0+1)^*$.

\noindent From here the value of $X$ after the substitution is

$X=\varepsilon + (0+1)^*1$, 

\noindent 
which is equivalent to the expression obtained using the other methods.

\subsubsection{Associating finite automata to regular expressions.}

Associate to the regular expression  $r$ a generalized finite automaton:

\smallskip
\begin{center}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(40.00,9.00)
\put(8.00,5.00){\circle{8.00}}
\put(36.00,5.00){\circle{8.00}}
\put(12.00,5.00){\vector(1,0){20.00}}
\put(36.00,5.00){\circle{6.00}}
\put(1.00,5.00){\vector(1,0){3.00}}
\put(22.00,7.00){\makebox(0,0)[cc]{$r$}}
\end{picture}
\end{center}


After this, use the transformations in Fig. \ref{KZ4kw} step by step, 
until an automaton with labels equal to letters from  $\Sigma$ or $\varepsilon$ will be obtained.

\begin{pelda}
Get started from regular expression $\varepsilon +(0+1)^*1$. The steps of transformations are in Fig.  
\ref{KZhosszu}(a)-(e). The last finite automaton (see Fig. \ref{KZhosszu}(e)) 
can be done in a simpler form as can be seen in Fig. \ref{KZhosszu}(f). 
After eliminating the $\varepsilon$-moves and transforming in a deterministic 
finite automaton the DFA in Fig.  \ref{KZhosszuveg} will be obtained, which 
is equivalent to DFA in Fig. \ref{KZ4ka1}.  
\end{pelda}


\begin{figure}[t!]
\hspace*{0.5cm}
\unitlength=1mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(110.00,14.00)
\put(10.00,10.00){\circle{8.00}}
\put(35.00,10.00){\circle{8.00}}
\put(50.00,10.00){\makebox(0,0)[cc]{replaced by }}
\put(63.00,10.00){\circle{8.00}}
\put(85.00,10.00){\circle{8.00}}
\put(106.00,10.00){\circle{8.00}}
\put(14.00,10.00){\vector(1,0){17.00}}
\put(67.00,10.00){\vector(1,0){14.00}}
\put(89.00,10.00){\vector(1,0){13.00}}
\put(23.00,13.00){\makebox(0,0)[cc]{$xy$}}
\put(75.00,13.00){\makebox(0,0)[cc]{$x$}}
\put(95.00,13.00){\makebox(0,0)[cc]{$y$}}
\end{picture}

\hspace*{0.5cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(89.00,21.00)
\put(10.00,10.00){\circle{8.00}}
\put(35.00,10.00){\circle{8.00}}
\put(50.00,10.00){\makebox(0,0)[cc]{replaced by }}
\put(63.00,10.00){\circle{8.00}}
\put(85.00,10.00){\circle{8.00}}
\put(14.00,10.00){\vector(1,0){17.00}}
\put(23.00,13.00){\makebox(0,0)[cc]{$x+y$}}
\bezier{92}(66.00,13.00)(74.00,21.00)(82.00,13.00)
\bezier{96}(66.00,7.00)(73.00,-2.00)(82.00,7.00)
\put(81.00,6.00){\vector(1,1){1.00}}
\put(81.00,14.00){\vector(1,-1){1.00}}
\put(74.00,20.00){\makebox(0,0)[cc]{$x$}}
\put(74.00,5.00){\makebox(0,0)[cc]{$y$}}
\end{picture}

\hspace*{0.5cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(110.00,23.00)
\put(10.00,10.00){\circle{8.00}}
\put(35.00,10.00){\circle{8.00}}
\put(50.00,10.00){\makebox(0,0)[cc]{replaced by }}
\put(63.00,10.00){\circle{8.00}}
\put(85.00,10.00){\circle{8.00}}
\put(106.00,10.00){\circle{8.00}}
\put(14.00,10.00){\vector(1,0){17.00}}
\put(67.00,10.00){\vector(1,0){14.00}}
\put(89.00,10.00){\vector(1,0){13.00}}
\put(23.00,13.00){\makebox(0,0)[cc]{$x^*$}}
\put(75.00,13.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(95.00,13.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(85.00,13.00){\oval(6.00,12.00)[t]}
\put(88.00,14.00){\vector(0,-1){1.00}}
\put(85.00,23.00){\makebox(0,0)[cc]{$x$}}
\end{picture}
\caption{Possible transformations to obtain finite automaton associated to a regular expression.}
\label{KZ4kw}
\vspace{-8mm}
\end{figure}



\begin{figure}
\noindent
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(47.00,21.00)
\put(8.00,17.00){\circle{8.00}}
\put(43.00,17.00){\circle{8.00}}
\put(1.00,17.00){\vector(1,0){3.00}}
\put(12.00,17.00){\vector(1,0){27.00}}
\put(43.00,17.00){\circle{6.00}}
\put(26.00,20.00){\makebox(0,0)[cc]{$\varepsilon + (0+1)^*1$}}
\put(25.00,1.00){\makebox(0,0)[cc]{(a)}}
\end{picture}
\quad\quad 
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(49.00,29.00)
\put(10.00,17.00){\circle{8.00}}
\put(45.00,17.00){\circle{8.00}}
\put(3.00,17.00){\vector(1,0){3.00}}
\put(45.00,17.00){\circle{6.00}}
\bezier{140}(13.00,20.00)(28.00,29.00)(42.00,20.00)
\bezier{136}(13.00,14.00)(28.00,6.00)(42.00,14.00)
\put(41.20,20.60){\vector(2,-1){1.00}}
\put(41.20,13.50){\vector(2,1){1.00}}
\put(28.00,27.30){\makebox(0,0)[cc]{$\varepsilon$}}
\put(28.00,6.00){\makebox(0,0)[cc]{$(0+1)^*1$}}
\put(28.00,1.00){\makebox(0,0)[cc]{(b)}}
\end{picture}

\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(60.00,42.00)
\put(8.00,17.00){\circle{8.00}}
\put(1.00,17.00){\vector(1,0){3.00}}
\put(33.00,17.00){\circle{8.00}}
\put(56.00,17.00){\circle{8.00}}
\put(12.00,17.00){\vector(1,0){17.00}}
\put(37.00,17.00){\vector(1,0){15.00}}
\bezier{248}(10.50,20.00)(35.00,42.00)(53.00,20.00)
\put(52.20,21.00){\vector(1,-1){1.00}}
\put(33.00,35.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(21.00,14.00){\makebox(0,0)[cc]{$(0+1)^*$}}
\put(44.00,14.00){\makebox(0,0)[cc]{1}}
\put(56.00,17.00){\circle{6.00}}
\put(32.00,1.00){\makebox(0,0)[cc]{(c)}}
\end{picture} 
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(60.00,41.00)
\put(8.00,17.00){\circle{8.00}}
\put(1.00,17.00){\vector(1,0){3.00}}
\put(23.00,17.00){\circle{8.00}}
\put(40.00,17.00){\circle{8.00}}
\put(56.00,17.00){\circle{8.00}}
\put(31.00,34.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(23.00,14.10){\oval(6.00,17.00)[b]}
\put(26.00,12.40){\vector(0,1){2.00}}
\put(15.00,14.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(31.00,14.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(47.00,14.00){\makebox(0,0)[cc]{1}}
\put(23.00,3.00){\makebox(0,0)[cc]{$0+1$}}
\put(56.00,17.00){\circle{6.00}}
\put(12.00,17.00){\vector(1,0){7.00}}
\put(27.00,17.00){\vector(1,0){9.00}}
\put(44.00,17.00){\vector(1,0){8.00}}
\bezier{240}(10.40,20.00)(33.00,41.00)(53.00,20.00)
\put(52.00,21.00){\vector(1,-1){1.00}}
\put(32.00,1.00){\makebox(0,0)[cc]{(d)}}
\end{picture}

\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(70.00,49.00)
\put(10.00,22.00){\circle{8.00}}
\put(3.00,22.00){\vector(1,0){3.00}}
\put(28.00,22.00){\circle{8.00}}
\put(46.00,22.00){\circle{8.00}}
\put(66.00,22.00){\circle{8.00}}
\put(14.00,22.00){\vector(1,0){10.00}}
\put(32.00,22.00){\vector(1,0){10.00}}
\put(50.00,22.00){\vector(1,0){12.00}}
\put(28.00,18.50){\oval(6.00,17.00)[b]}
\put(31.00,17.00){\vector(0,1){2.00}}
\put(19.00,19.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(37.00,19.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(56.00,19.00){\makebox(0,0)[cc]{1}}
\put(28.00,13.00){\makebox(0,0)[cc]{$0$}}
\put(28.00,25.00){\oval(6.00,18.00)[t]}
\put(31.00,27.00){\vector(0,-1){2.00}}
\put(28.00,31.00){\makebox(0,0)[cc]{1}}
\put(65.00,27.00){\vector(1,-1){1.00}}
\put(39.00,34.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(66.00,22.00){\circle{6.00}}
\bezier{288}(10.00,26.00)(39.00,49.00)(66.00,26.00)
\put(40.00,1.00){\makebox(0,0)[cc]{(e)}}
\end{picture}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(50.00,37.00)
\put(10.00,22.00){\circle{8.00}}
\put(3.00,22.00){\vector(1,0){3.00}}
\put(28.00,22.00){\circle{8.00}}
\put(46.00,22.00){\circle{8.00}}
\put(14.00,22.00){\vector(1,0){10.00}}
\put(32.00,22.00){\vector(1,0){10.00}}
\put(28.00,18.50){\oval(6.00,17.00)[b]}
\put(31.00,17.00){\vector(0,1){2.00}}
\put(19.00,19.00){\makebox(0,0)[cc]{$\varepsilon$}}
\put(37.00,19.00){\makebox(0,0)[cc]{$1$}}
\put(28.00,7.00){\makebox(0,0)[cc]{$0$}}
\put(28.00,25.00){\oval(6.00,18.00)[t]}
\put(31.00,27.00){\vector(0,-1){2.00}}
\put(28.00,37.00){\makebox(0,0)[cc]{1}}
\put(46.00,22.00){\circle{6.00}}
\put(10.00,22.00){\circle{6.00}}
\put(28.00,1.00){\makebox(0,0)[cc]{(f)}}
\end{picture}
\caption{Associating finite automaton to regular expression  $\varepsilon +(0+1)^*1$.}
\label{KZhosszu}
\end{figure}

\begin{figure}\centering
\unitlength=1mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(64.00,50.00)
\put(10.00,24.00){\circle{8.00}}
\put(35.00,24.00){\circle{8.00}}
\put(60.00,24.00){\circle{8.00}}
\put(10.00,24.00){\circle{6.00}}
\put(60.00,24.00){\circle{6.00}}
\put(2.00,24.00){\vector(1,0){4.00}}
\put(14.00,24.00){\vector(1,0){17.00}}
\put(39.00,24.00){\vector(1,0){17.00}}
\put(35.00,21.00){\oval(6.00,18.00)[b]}
\put(60.00,21.00){\oval(6.00,18.00)[b]}
\put(38.00,19.00){\vector(0,1){2.00}}
\put(63.00,19.00){\vector(0,1){2.00}}
\bezier{104}(57.00,27.00)(47.00,36.00)(38.00,27.00)
\put(39.00,28.00){\vector(-1,-1){1.00}}
\bezier{268}(60.00,28.00)(38.00,50.00)(10.00,28.00)
\put(59.00,29.00){\vector(1,-1){1.00}}
\put(36.00,43.00){\makebox(0,0)[cc]{1}}
\put(47.00,29.00){\makebox(0,0)[cc]{0}}
\put(47.00,21.00){\makebox(0,0)[cc]{1}}
\put(20.00,21.00){\makebox(0,0)[cc]{0}}
\put(35.00,9.00){\makebox(0,0)[cc]{0}}
\put(60.00,9.00){\makebox(0,0)[cc]{1}}
\end{picture}
\caption{Finite automaton associated  to regular expression $\varepsilon +(0+1)^*1$.}
\label{KZhosszuveg}
\end{figure}


\begin{gyak}
\ujgyak
Give a DFA which accepts natural numbers divisible by 9.
\ujgyak
Give a DFA which accepts the language containing all words formed by   

\textit{a.} an even number of 0's and an even number of 1's, 

\textit{b.} an even number of 0's and an odd number of 1's, 

\textit{c.}  an odd number of 0's and an even number of 1's,

\textit{d.}  an odd number of 0's and an odd number of 1's. 
\ujgyak
Give a DFA\gyakindex{DFA} to accept respectively the following languages:

$L_1= \{ a^nb^m\; |\; n\ge 1, m\ge 0\}, \qquad L_2= \{ a^nb^m\; |\; n\ge 1, m\ge 1\},$

$L_3= \{ a^nb^m\; |\; n\ge 0, m\ge 0\}, \qquad L_4= \{ a^nb^m\; |\; n\ge 0, m\ge 1\}.$
\ujgyak
Give an NFA\gyakindex{NFA} which accepts words containing at least two  0's and any number of 1's. Give an equivalent DFA.
\ujgyak\label{KZgygy}
Minimize the DFA's in Fig. \ref{KZ20.2-5}. 
\begin{figure}[t!]
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(112.00,39.00)
\put(10.00,12.00){\circle{8.00}}
\put(40.00,12.00){\circle{8.00}}
\put(10.00,35.00){\circle{8.00}}
\put(40.00,35.00){\circle{8.00}}
\put(14.00,35.00){\vector(1,0){22.00}}
\put(37.80,31.80){\vector(-3,-2){25.20}}
\put(40.00,16.00){\vector(0,1){15.00}}
\put(40.00,35.00){\circle{6.00}}
\put(40.00,12.00){\circle{6.00}}
\bezier{80}(7.60,15.60)(2.00,23.00)(8.00,31.00)
\bezier{72}(12.00,31.00)(17.00,24.00)(12.00,16.00)
\bezier{104}(13.80,14.00)(25.00,20.00)(36.60,14.20)
\bezier{108}(13.00,9.00)(25.00,3.00)(37.00,9.00)
\put(35.60,14.80){\vector(2,-1){1.00}}
\put(14.00,8.60){\vector(-2,1){1.00}}
\put(10.00,35.00){\makebox(0,0)[cc]{$p$}}
\put(40.00,35.00){\makebox(0,0)[cc]{$q$}}
\put(10.00,12.00){\makebox(0,0)[cc]{$r$}}
\put(40.00,12.00){\makebox(0,0)[cc]{$s$}}
\put(43.20,35.00){\oval(13.00,6.00)[r]}
\put(45.00,38.00){\vector(-1,0){2.00}}
\put(52.00,35.00){\makebox(0,0)[cc]{$a$}}
\put(26.00,38.00){\makebox(0,0)[cc]{$a$}}
\put(3.00,24.00){\makebox(0,0)[cc]{$b$}}
\put(12.00,24.00){\makebox(0,0)[cc]{$a$}}
\put(30.00,24.00){\makebox(0,0)[cc]{$b$}}
\put(42.00,24.00){\makebox(0,0)[cc]{$a$}}
\put(25.00,14.00){\makebox(0,0)[cc]{$b$}}
\put(25.00,3.00){\makebox(0,0)[cc]{$b$}}
\put(70.00,12.00){\circle{8.00}}
\put(100.00,12.00){\circle{8.00}}
\put(70.00,35.00){\circle{8.00}}
\put(100.00,35.00){\circle{8.00}}
\put(63.00,35.00){\vector(1,0){3.00}}
\put(74.00,35.00){\vector(1,0){22.00}}
\put(97.80,31.80){\vector(-3,-2){25.20}}
\put(100.00,16.00){\vector(0,1){15.00}}
\put(100.00,35.00){\circle{6.00}}
\put(100.00,12.00){\circle{6.00}}
\bezier{80}(67.80,15.40)(62.00,23.00)(68.00,31.00)
\bezier{72}(72.00,31.00)(77.00,24.00)(72.00,16.00)
\put(70.00,35.00){\makebox(0,0)[cc]{$1$}}
\put(100.00,35.00){\makebox(0,0)[cc]{$2$}}
\put(70.00,12.00){\makebox(0,0)[cc]{$3$}}
\put(100.00,12.00){\makebox(0,0)[cc]{$4$}}
\put(103.20,35.00){\oval(13.00,6.00)[r]}
\put(105.00,38.00){\vector(-1,0){2.00}}
\put(112.00,35.00){\makebox(0,0)[cc]{$a$}}
\put(86.00,38.00){\makebox(0,0)[cc]{$a$}}
\put(63.00,24.00){\makebox(0,0)[cc]{$b$}}
\put(72.00,24.00){\makebox(0,0)[cc]{$a$}}
\put(90.00,24.00){\makebox(0,0)[cc]{$b$}}
\put(102.00,24.00){\makebox(0,0)[cc]{$a$}}
\put(85.00,14.00){\makebox(0,0)[cc]{$b$}}
\put(112.00,12.00){\makebox(0,0)[cc]{$b$}}
\put(103.20,12.00){\oval(13.00,6.00)[r]}
\put(10.00,12.00){\circle{6.00}}
\put(74.00,11.00){\vector(1,0){22.00}}
\put(6.80,17.20){\vector(1,-2){1.00}}
\put(13.00,29.30){\vector(-1,2){1.00}}
\put(73.00,29.30){\vector(-1,2){1.00}}
\put(66.80,17.20){\vector(1,-2){1.00}}
\put(104.00,15.00){\vector(-1,0){1.00}}
\end{picture}
\caption{DFA's to minimize for Exercise \ref{KZgygy}}
\label{KZ20.2-5}
\end{figure}
\ujgyak\label{KZgy0}
 Show that the DFA in  \ref{KZfel24}.(a) is a minimum state automaton.
\noindent\begin{figure}[t!]
\begin{minipage}{6.5cm}
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(54.00,44.00)
\put(10.00,10.00){\circle{8.00}}
\put(50.00,10.00){\circle{8.00}}
\put(30.00,36.00){\circle{8.00}}
\put(10.00,10.00){\circle{6.00}}
\put(50.00,10.00){\circle{6.00}}
\put(30.00,44.00){\vector(0,-1){4.00}}
\bezier{148}(13.00,7.00)(30.00,0.00)(47.00,7.00)
\bezier{156}(13.00,13.00)(30.00,22.00)(47.50,13.00)
\bezier{108}(12.00,13.00)(28.00,24.00)(28.00,32.00)
\bezier{132}(26.00,36.00)(9.00,28.00)(9.00,14.00)
\bezier{140}(34.00,36.00)(51.00,31.00)(52.00,14.00)
\bezier{104}(48.40,14.00)(32.00,25.00)(32.00,32.00)
\put(9.00,16.00){\vector(0,-1){2.00}}
\put(28.00,31.40){\vector(0,1){1.00}}
\put(32.00,31.50){\vector(0,1){1.00}}
\put(52.00,14.50){\vector(0,-1){1.00}}
\put(46.40,13.60){\vector(2,-1){1.00}}
\put(14.00,6.50){\vector(-2,1){1.00}}
\put(30.00,14.00){\makebox(0,0)[cc]{$b$}}
\put(30.00,6.00){\makebox(0,0)[cc]{$a$}}
\put(11.00,27.00){\makebox(0,0)[cc]{$a$}}
\put(23.00,24.00){\makebox(0,0)[cc]{$a$}}
\put(38.00,24.00){\makebox(0,0)[cc]{$b$}}
\put(50.00,27.00){\makebox(0,0)[cc]{$b$}}
\put(30.00,36.00){\makebox(0,0)[cc]{{\small $q_0$}}}
\put(10.00,10.00){\makebox(0,0)[cc]{{\small $q_1$}}}
\put(50.00,10.00){\makebox(0,0)[cc]{{\small $q_2$}}}
\end{picture}
\\
\centerline{(a)}
\end{minipage}
\begin{minipage}{6.5cm}

\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(54.00,28.00)
\put(10.00,15.00){\circle{8.00}}
\put(30.00,15.00){\circle{8.00}}
\put(50.00,15.00){\circle{8.00}}
\put(10.00,15.00){\circle{6.00}}
\put(50.00,15.00){\circle{6.00}}
\put(2.00,15.00){\vector(1,0){4.00}}
\put(14.00,15.00){\vector(1,0){12.00}}
\put(34.00,15.00){\vector(1,0){12.00}}
\put(50.00,17.50){\oval(6.00,15.00)[t]}
\put(30.00,17.50){\oval(6.00,15.00)[t]}
\put(10.00,17.50){\oval(6.00,15.00)[t]}
\put(13.00,20.00){\vector(0,-1){2.00}}
\put(33.00,20.00){\vector(0,-1){2.00}}
\put(53.00,20.00){\vector(0,-1){2.00}}
\put(10.00,28.00){\makebox(0,0)[cc]{1}}
\put(30.00,28.00){\makebox(0,0)[cc]{0}}
\put(50.00,28.00){\makebox(0,0)[cc]{1}}
\put(19.00,17.00){\makebox(0,0)[cc]{0,1}}
\bezier{180}(10.00,11.00)(30.00,1.00)(50.00,11.00)
\put(30.00,3.00){\makebox(0,0)[cc]{0,1}}
\put(49.00,10.50){\vector(2,1){1.00}}
\put(10.00,15.00){\makebox(0,0)[cc]{$p$}}
\put(30.00,15.00){\makebox(0,0)[cc]{$q$}}
\put(50.00,15.00){\makebox(0,0)[cc]{$r$}}
\put(40.00,17.00){\makebox(0,0)[cc]{0,1}}
\end{picture}

\vspace*{1cm}\centerline{(b)}
\end{minipage}
\caption{Finite automata for Exercises  \ref{KZgy0}  and  \ref{KZgy1}.}
\label{KZfel24}
\end{figure}
\ujgyak\label{KZgy1}
Transform NFA  in Fig. \ref{KZfel24}.(b) in a DFA, and after this minimize it.
\ujgyak
Define finite automaton ${\mathrm A_1}$ which accepts all words of the form $0(10)^n$  ($n\ge 0$), and finite automaton ${\mathrm A_2}$ which accepts all words of the form $1(01)^n$ ($n\ge 0$). Define the union automaton  ${\mathrm A_1}\cup {\mathrm A_2}$, and then  eliminate the  $\varepsilon$-moves.
\ujgyak\label{KZgy2}
Associate to DFA in Fig.  \ref{KZvr} a regular expression.
\begin{figure}\centering
\unitlength=1mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(72.00,23.00)
\put(10.00,10.00){\circle{8.00}}
\put(29.00,10.00){\circle{8.00}}
\put(49.00,10.00){\circle{8.00}}
\put(68.00,10.00){\circle{8.00}}
\put(2.00,10.00){\vector(1,0){4.00}}
\put(14.00,10.00){\vector(1,0){11.00}}
\put(33.00,10.00){\vector(1,0){12.00}}
\put(53.00,10.00){\vector(1,0){11.00}}
\put(10.00,13.00){\oval(6.00,14.00)[t]}
\put(68.00,13.00){\oval(6.00,14.00)[t]}
\put(13.00,14.00){\vector(0,-1){1.00}}
\put(71.00,14.00){\vector(0,-1){1.00}}
\put(10.00,23.00){\makebox(0,0)[cc]{0,1}}
\put(19.00,12.00){\makebox(0,0)[cc]{1}}
\put(38.00,12.00){\makebox(0,0)[cc]{0}}
\put(58.00,12.00){\makebox(0,0)[cc]{1}}
\put(68.00,23.00){\makebox(0,0)[cc]{0,1}}
\put(10.00,10.00){\makebox(0,0)[cc]{{\small $q_0$}}}
\put(29.00,10.00){\makebox(0,0)[cc]{{\small $q_1$}}}
\put(49.00,10.00){\makebox(0,0)[cc]{{\small $q_2$}}}
\put(68.00,10.00){\makebox(0,0)[cc]{{\small $q_3$}}}
\put(68.00,10.00){\circle{6.00}}
\end{picture}
\caption{DFA for Exercise  \ref{KZgy2}.}
\label{KZvr}
\end{figure}
\ujgyak
Associate to regular expression  $ab^*ba^*+b+ba^*a$ a DFA.
\ujgyak
Prove, using the pumping lemma, that none of the following languages are regular:

$L_1= \big\{a^ncb^n\;|\; n\ge 0  \big\}, \qquad L_2= \big\{a^nb^na^n\;|\; n\ge 0 \big\},  
\qquad L_3= \big\{a^p\;|\; p \textrm{ prím}   \big\}.$
\ujgyak
Prove that if $L$ is a regular language, then   $\{ u^{-1}\;|\; u\in L\}$ is also regular.
\ujgyak
Prove that if  $L\subseteq \Sigma^*$ is a regular language, then  the following languages are also regular.

$\textrm{pre}(L)= \{w\in \Sigma^*\;|\; \exists u\in \Sigma^*, wu\in L \}, \;  
 \textrm{suf}(L)= \{w\in \Sigma^*\;|\; \exists u\in \Sigma^*, uw\in L\}. $
 

\ujgyak
 Show that the following languages are all regular.
 
$L_1=\{ab^ncd^m\;|\; n>0, m>0\}, $

$L_2=\{(ab)^n \;|\; n\ge 0\},$ 

$L_3=\{a^{kn}\;|\; n\ge 0, \;\; k \textrm{ constant}\}.$
\end{gyak}


\section{Pushdown automata and context-free languages}\index{pushdown automata}
In this \index{grammar!context-free} section we deal with the pushdown automata 
and the class of languages --- the context-free languages --- accepted by them.

As we have been seen in Section \ref{KZalapfogalmak}, a context-free grammar  
$G=(N,T,P,S)$ is one with the productions of the form  $A\rightarrow \beta$, $A\in N$, $\beta\in (N\cup T)^+.$  
The production $S\rightarrow \varepsilon$ is also permitted if $S$ does not appear in right hand  side of any productions. 
Language $L(G)= \{u\in T^*\;|\; S\overset{*}{\underset{G}\Longrightarrow} u \}$
is the context-free language generated by grammar $G$.

\subsection{Pushdown automata}
We have been seen that finite automata accept the class of regular languages. Now we get to know a new kind of automata, the so-called  \ki{pushdown automata,}\inddef{automata!pushdown} which accept context-free languages. 
The pushdown automata differ from finite automata mainly in that to  have the possibility to change states without reading any input symbol (i.e. to read the empty symbol) and possess a stack memory, which uses the so-called stack symbols (See Fig. \ref{KZverem1}). 
 
\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(106.00,49.00)
\emline{60.00}{49.00}{1}{5.00}{49.00}{2}
\emline{5.00}{49.00}{3}{5.00}{39.00}{4}
\emline{5.00}{39.00}{5}{60.00}{39.00}{6}
\emline{15.00}{39.00}{7}{15.00}{49.00}{8}
\emline{25.00}{49.00}{9}{25.00}{39.00}{10}
\emline{35.00}{39.00}{11}{35.00}{49.00}{12}
\emline{45.00}{49.00}{13}{45.00}{39.00}{14}
\emline{55.00}{39.00}{15}{55.00}{49.00}{16}
\emline{25.00}{29.00}{17}{66.00}{29.00}{18}
\emline{66.00}{29.00}{19}{66.00}{9.00}{20}
\emline{66.00}{9.00}{21}{25.00}{9.00}{22}
\emline{25.00}{9.00}{23}{25.00}{29.00}{24}
\emline{88.00}{38.00}{25}{88.00}{3.00}{26}
\emline{88.00}{3.00}{27}{98.00}{3.00}{28}
\emline{98.00}{3.00}{29}{98.00}{38.00}{30}
\emline{98.00}{13.00}{31}{88.00}{13.00}{32}
\emline{88.00}{23.00}{33}{98.00}{23.00}{34}
\emline{98.00}{33.00}{35}{88.00}{33.00}{36}
\emline{88.00}{38.00}{37}{88.00}{48.00}{38}
\emline{98.00}{48.00}{39}{98.00}{36.00}{40}
\emline{98.00}{43.00}{41}{88.00}{43.00}{42}
\emline{20.00}{39.00}{43}{20.00}{34.00}{44}
\emline{20.00}{34.00}{45}{46.00}{34.00}{46}
\emline{46.00}{34.00}{47}{46.00}{29.00}{48}
\put(20.00,37.00){\vector(0,1){2.00}}
\emline{46.00}{9.00}{49}{46.00}{3.00}{50}
\emline{46.00}{3.00}{51}{73.00}{3.00}{52}
\emline{73.00}{3.00}{53}{73.00}{16.00}{54}
\put(61.00,48.00){\makebox(0,0)[lt]{input}}
\put(10.00,44.00){\makebox(0,0)[cc]{$a_1$}}
\put(20.00,44.00){\makebox(0,0)[cc]{$a_2$}}
\put(30.00,44.00){\makebox(0,0)[cc]{\ldots}}
\put(40.00,44.00){\makebox(0,0)[cc]{$a_n$}}
\put(46.00,19.00){\makebox(0,0)[cc]{control unit}}
\put(106.00,45.00){\makebox(0,0)[cc]{stack}}
\put(93.00,8.00){\makebox(0,0)[cc]{$z_0$}}
\put(93.00,18.00){\makebox(0,0)[cc]{$z_1$}}
\put(93.00,28.00){\makebox(0,0)[cc]{\ldots}}
\put(93.00,38.00){\makebox(0,0)[cc]{$z_k$}}
\emline{25.00}{19.00}{55}{14.00}{19.00}{56}
\put(11.00,19.00){\makebox(0,0)[rc]{\small yes/no}}
\put(15.00,19.00){\vector(-1,0){2.00}}
\emline{73.00}{16.00}{57}{73.00}{38.00}{58}
\put(61.00,40.00){\makebox(0,0)[lb]{tape}}
\put(73.00,38.00){\vector(1,0){15.00}}
\end{picture}
\caption{Pushdown automaton.}
\label{KZverem1}
\end{figure}


The pushdown automaton get a word as input, start to function from an initial state having in the 
stack a special symbol, the initial stack symbol. While working, the pushdown automaton change its state 
based on current state, next input symbol (or empty word) and stack top symbol and replace the top 
symbol in the stack with a (possibly empty) word.
 
There are two type of acceptances.  The pushdown automaton accepts a word by final state when after reading it the automaton enter a final state.  The pushdown automaton accepts a word by empty stack when after reading it the automaton empties its stack.  We show that these two acceptances are equivalent.

\begin{defi} A \ki{nondeterministic pushdown automaton}\inddef{nondeterministic pushdown automaton}\inddef{automata!nondeterministic pushdown} is a system
 
\centerline{${\mathrm V}=(Q,\Sigma,W,E,q_0,z_0,F)$ \koz ,} 

\noindent where

$\bullet$ $Q$ is the finite, non-empty set of \ki{states}\inddef{state!of pushdown automaton}
 
$\bullet$ $\Sigma$ is the \ki{input alphabet,}\inddef{input alphabet!of pushdown automaton}

$\bullet$ $W$ is the  \ki{stack alphabet,}\inddef{stack alphabet!of pushdown automaton}

$\bullet$ $E\subseteq Q\times\big( \Sigma\cup \{\varepsilon\}  \big)\times W\times W^*\times Q$ is the set of   \ki{transitions} or \ki{edges},\inddef{transition!of pushdown automata}\index{edge!of pushdown automata|see{transition}}

$\bullet$ $q_0\in Q$ is the  \ki{initial state,}\inddef{initial state!of pushdown automata}

$\bullet$ $z_0\in W$ is the  \ki{start symbol of stack,}\inddef{start symbol!of pushdown automata}

$\bullet$ $F\subseteq Q$ is the set of  \ki{final states}.\inddef{final state!of pushdown automata}
\end{defi}


A transition  $(p,a,z,w,q)$ means that if pushdown automaton  ${\mathrm V}$ is in state $p$, 
reads from the input tape letter  $a$  (instead of input letter we can also consider the empty word $\varepsilon$), and the top symbol in the stack is  $z$, then the pushdown automaton enters state $q$ and replaces in the stack $z$  by word $w$. Writing word  $w$ in the stack is made by natural order (letters of word $w$ will be put in the stack letter by letter from left to right). Instead of writing transition $(p,a,z,w,q)$ we will use a more suggestive notation  $\big(p,(a,z/w),q\big)$.

Here, as in the case of finite automata, we can define a transition function
\[\delta\;:\; Q\times (\Sigma\cup \{\varepsilon\})\times W\rightarrow 
     {\mathcal P}(W^*\times Q) \koz , \]
which associate to current state, input letter and top letter in stack pairs of the form $(w,q)$, where $w\in W^*$ is the word written in stack and  $q\in Q$ the new state.

Because the pushdown automaton is nondeterministic, we will have for the transition function 

$\delta (q,a,z)=\{ (w_1,p_1), \ldots , ( w_k,p_k) \}$ (if the pushdown automaton reads an input letter and moves to right), or 

$\delta (q,\varepsilon,z)=\{ (w_1,p_1), \ldots , (w_k,p_k) \}$
(without move on the input tape).

A pushdown  automaton is  \ki{deterministic,}\inddef{pushdown automata!deterministic} \inddef{deterministic pushdown automata} if for any  $q\in Q$  and  $z \in W$ we have
    
    $\bullet$  $|\delta(q,a,z)|\le 1$, $\forall a  \in \Sigma\cup \{\varepsilon\} $ and 
    
    $\bullet$ if $\delta(q,\varepsilon,z)\not=\emptyset$, then  $\delta(q,a,z)=\emptyset$, $\forall a\in \Sigma$.

We can associate to any pushdown automaton a transition table, exactly as in the case of finite automata.
The rows of this table are indexed by elements of $Q$,  the columns by elements from $\Sigma\cup \{\varepsilon\}$ and $W$  (to each $a\in\Sigma\cup \{\varepsilon\}$ and  $z\in W$ will correspond a column). At intersection of row corresponding to state  $q\in Q$  and column corresponding to  $a\in\Sigma\cup \{\varepsilon\}$ and $z\in W$ we will  have pairs $(w_1,p_1), \ldots, (w_k,p_k)$ if  $\delta (q,a,z)=\{ (w_1,p_1), \ldots , ( w_k,p_k) \}$. 
 
The transition graph, in which the label of edge  $(p, q)$ will be  $(a,z/w)$ corresponding to transition 
 $\big(p,(a,z/w),q\big)$, can be also defined.  
 
\begin{figure}[t!]\centering
{\small 
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(64.00,44.00)
\put(10.00,25.00){\circle{10.00}}
\put(50.00,10.00){\circle{10.00}}
\put(50.00,39.00){\circle{10.00}}
\put(0.00,25.00){\vector(1,0){5.00}}
\put(14.00,29.00){\vector(4,1){31.00}}
\put(54.00,39.50){\oval(16.00,7.00)[r]}
\put(54.00,9.50){\oval(16.00,7.00)[r]}
\put(55.00,43.00){\vector(-1,0){2.00}}
\put(55.00,6.00){\vector(-1,0){2.00}}
\put(10.00,25.00){\makebox(0,0)[cc]{$q_0$}}
\put(50.00,39.00){\makebox(0,0)[cc]{$q_1$}}
\put(50.00,10.00){\makebox(0,0)[cc]{$q_2$}}
\put(45.00,9.00){\vector(-2,1){30.00}}
\put(50.00,34.00){\vector(0,-1){19.00}}
\put(10.00,25.00){\circle{8.00}}
\put(42.00,38.00){\makebox(0,0)[rc]{$(a,z_0/z_0z_1)$}}
\put(37.00,11.00){\makebox(0,0)[rc]{$(\varepsilon, z_0/\varepsilon)$}}
\put(52.00,25.00){\makebox(0,0)[lc]{$(b,z_1/\varepsilon)$}}
\put(64.00,8.00){\makebox(0,0)[lc]{$(b,z_1/\varepsilon)$}}
\put(64.00,39.00){\makebox(0,0)[lc]{$(a,z_1/z_1z_1)$}}
\end{picture}
}
\caption{Example of pushdown automaton.}
\label{KZ6ka2}
\end{figure}

  
\begin{pelda}\label{KZelsopelda}
${\mathrm V}_1=(\{q_0,q_1,q_2\}, \{a,b\}, \{z_0,z_1\}, E, q_0, z_0, \{q_0\})$. Elements of  $E$ are:

\begin{tabbing}
\qquad \= $\big(q_0, (a, z_0/z_0z_1), q_1 \big)$ \hspace*{2cm}\= \\
\> $\big(q_1, (a, z_1/z_1z_1), q_1 \big)$  \>    $\big(q_1, (b, z_1/\varepsilon), q_2 \big)$\\
\> $\big(q_2, (b, z_1/\varepsilon), q_2 \big)$  \> $\big(q_2, (\varepsilon, z_0/\varepsilon), q_0 \big) \koz .$
\end{tabbing}

\medskip
\noindent The transition function:

\begin{center}\begin{tabbing}
\hspace*{0.5cm} 
\= $\delta(q_0,a,z_0)= \{(z_0z_1,q_1)\}$  \hspace*{1.5cm}\=     \\
\> $\delta(q_1,a,z_1)= \{(z_1z_1,q_1)\}$ \>  $\delta(q_1,b,z_1)=\{(\varepsilon,q_2)\} $     \\
\> $\delta(q_2,b,z_1)=\{(\varepsilon,q_2)\}$    \>   $\delta(q_2,\varepsilon,z_0)=\{(\varepsilon,q_0)\} \koz .$    
\end{tabbing}
\end{center}

\medskip
\noindent The transition table:

\smallskip
\begin{center}
\begin{tabular}{|c||c|c|c|c|}\hline
$\Sigma\cup\{\varepsilon\}$ &\multicolumn{2}{c|}{$a$} &  $b$  & $\varepsilon$ \\ \hline
$W$ & $z_0$   & $z_1$  &    $z_1$  & $z_0$   \\ \hline\hline
      & & & &   \\
$q_0$ & $(z_0z_1,q_1)$  & & &  $\quad\qquad$\\ 
      & & & &   \\ \hline
      & & & &  \\
$q_1$ &   & $(z_1z_1,q_1)$ &  $( \varepsilon,q_2)$ &   \\ 
      & & & &   \\ \hline
      & & & &   \\
$q_2$ &  &  & $(\varepsilon,q_2)$ & $(\varepsilon,q_0)$   
\\  
      & & & &   \\ \hline
\end{tabular}
\end{center}

\medskip
\noindent Because for the transition function every set which is not empty contains only one element 
(e.g. $\delta(q_0,a,z_0)= \{(z_0z_1,q_1)\}$), in the above table each cell contains only one element, And the set notation is not used. Generally, if a set has more than one element, then its elements are written one under other. The transition graph of this pushdown automaton is in Fig.  \ref{KZ6ka2}.
\end{pelda}

The current state, the unread part of the input word and the content of stack constitutes a 
\ki{configuration}\inddef{configuration} of the pushdown automaton, i.e. for each  $ q\in Q$, $u\in \Sigma^*$ and $v\in W^*$  the triplet $(q,u,v)$ can be a configuration.
          
If $u=a_1a_2\ldots a_k$ and $v=x_1x_2\ldots x_m$, then the pushdown automaton can change its configuration in two ways:

\noindent 
$\bullet$  $(q, \;a_1a_2\ldots a_k, \;x_1x_2\ldots x_{m-1}x_m) \Longrightarrow 
    (p,\; a_2a_3\ldots a_k, \; x_1,x_2\ldots x_{m-1}w)$, 

if $\big(q,(a_1,x_m/w),p\big)\in E$

\noindent 
$\bullet$  $(q, \;a_1a_2\ldots a_k,\; x_1x_2\ldots x_m) \Longrightarrow (p, \;a_1a_2\ldots a_k, 
          \; x_1,x_2\ldots x_{m-1}w)$, 

if $\big(q, (\varepsilon, x_m/w),p\big)\in E.$

The reflexive and transitive closure of the relation  $\Longrightarrow$ will be denoted by  $\overset{*}{\Longrightarrow}$. Instead of using $\Longrightarrow$,  sometimes  $\vdash$  is considered.

How does work such a pushdown automaton? Getting started with the initial configuration $(q_0, a_1a_2\ldots a_n, z_0)$ 
we will consider all possible next configurations, and after this the next configurations to these next configurations, and so on, until it is possible. 

\begin{defi} Pushdown automaton  ${\mathrm V}$ accepts (recognizes) word $u$  \ki{by final state} if there exist a sequence of configurations of $\;{\mathrm V}$ for which the following are true:

$\bullet$  the first element of the sequence is  $(q_0,u,z_0)$,

$\bullet$  there is a going from each element of the sequence to the next element,  excepting the case when the sequence has only one element,

$\bullet$  the last element of the sequence is  $(p,\varepsilon, w)$, where $p\in F$ and $w \in W^*$. 
\end{defi}

Therefore pushdown automaton  ${\mathrm V}$ accepts word  $u$ by final state, if and only if 
$(q_0, u, z_0)\overset{*}{\Longrightarrow} (p,\varepsilon, w)$ for some $w\in W^*$ and $\;p\in F$. 
 The set of words accepted by final state by pushdown automaton ${\mathrm V}$ will be called the language accepted by  ${\mathrm V}$ by final state and will be denoted by $L({\mathrm V})$.

\begin{defi}  Pushdown automaton  ${\mathrm V}$ accepts (recognizes) word $u$  \ki{by empty stack} if there exist a sequence of configurations of $\;{\mathrm V}$ for which the following are true:

$\bullet$ the first element of the sequence is $(q_0,u,z_0)$,

$\bullet$ there is a going from each element of the sequence to the next element,

$\bullet$ the last element of the sequence is  $(p,\varepsilon, \varepsilon)$ and
$p$  is an arbitrary state.
\end{defi}

Therefore pushdown automaton  ${\mathrm V}$ accepts a word  $u$ by empty stack if $(q_0, u, z_0)\overset{*}{\Longrightarrow} (p,\varepsilon , \varepsilon)$ for some  $p\in Q$. The set of words accepted by empty stack by pushdown automaton  ${\mathrm V}$ will be called the language accepted by empty stack by ${\mathrm V}$ and will be denoted by  $L_{\varepsilon}({\mathrm V})$.

\begin{pelda}  Pushdown automaton ${\mathrm V}_1$ of Example  \ref{KZelsopelda} accepts the language   $\{a^nb^n\;|\; n\ge 0\}$ by final state. Consider the derivation for words  
$aaabbb$ and $abab$.

\smallskip
Word $a^3b^3$ is accepted by the considered pushdown automaton because

$(q_0,aaabbb, z_0) \Longrightarrow  (q_1, aabbb, z_0z_1)\Longrightarrow (q_1,abbb, z_0z_1z_1)\Longrightarrow (q_1, bbb, z_0z_1z_1z_1)$

$\Longrightarrow (q_2,bb,z_0z_1z_1)\Longrightarrow (q_2,b,z_0z_1)\Longrightarrow  (q_2,\varepsilon, z_0)\Longrightarrow (q_0, \varepsilon, \varepsilon) $ and because  $q_0$ is a final state the pushdown automaton accepts this word. But the stack being empty, it accepts this word also by empty stack.  

Because the initial state is also a final state, the empty word is accepted by final state, but not by empty stack. 

To show that word  $abab$ is not accepted, we need to study all possibilities.  It is easy to see that in our case 
there is only a single possibility:

$(q_0, abab, z_0)\Longrightarrow (q_1,bab, z_0z_1)\Longrightarrow (q_2, ab, z_0)\Longrightarrow (q_0, ab, \varepsilon)$, but there is no further going, so word  $abab$ is not accepted.
\end{pelda}

\begin{pelda}\label{KZmasodikpelda}
 The transition table of the pushdown automaton  ${\mathrm V}_2= (\{q_0,q_1\}, \{0,1\}, \{z_0,z_1,z_2\}, E, q_0,z_0, \emptyset )$ is: 

\begin{center}
\begin{tabular}{|c|c|c|c|c|c|c|c|}\hline
$\Sigma\cup\{\varepsilon\}$ & \multicolumn{3}{c|}{0} & \multicolumn{3}{c|}{1} & $\varepsilon$\\ \hline
$W$ &$z_0$&$z_1$&$z_2$&$z_0$&$z_1$&$z_2$&$z_0$ \\ \hline\hline
 & & & & & & &  \\
 $q_0$&$(z_0z_1,q_0 )$&$(z_1z_1,q_0)$&$(z_2z_1,q_0 )$&$(z_0z_2,q_0)$ 
       &$(z_1z_2,q_0)$& $(z_2z_2,q_0 )$  &$(\varepsilon,q_1)$ \\ 
  & &$(\varepsilon,q_1)$& & & &$(\varepsilon,q_1)$ &  \\
 & & & & & & &  \\ \hline
 & & & & & &  &\\
$q_1$& &$(\varepsilon,q_1)$&& & &$(\varepsilon,q_1)$&$(\varepsilon,q_1)$\\ 
 & & & & & & &  \\ \hline
\end{tabular}\end{center}

\begin{figure}[t!]\centering
\bigskip
{\small 
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(82.00,59.00)
\put(19.00,35.00){\circle{14.00}}
\put(19.00,35.00){\circle{14.00}}
\put(75.00,35.00){\circle{14.00}}
\put(19.00,38.50){\oval(12.00,21.00)[t]}
\put(75.00,38.50){\oval(12.00,21.00)[t]}
\put(75.00,30.50){\oval(12.00,21.00)[b]}
\put(19.00,30.50){\oval(12.00,21.00)[b]}
\put(25.00,41.00){\vector(0,-1){2.00}}
\put(25.00,28.00){\vector(0,1){3.00}}
\put(81.00,41.00){\vector(0,-1){2.00}}
\put(81.00,29.00){\vector(0,1){2.00}}
\put(5.00,36.00){\vector(1,0){7.00}}
\put(19.00,36.00){\makebox(0,0)[cc]{$q_0$}}
\put(75.00,36.00){\makebox(0,0)[cc]{$q_1$}}
\put(48.00,32.00){\makebox(0,0)[cc]{$(0,z_1/\varepsilon)$}}
\put(48.00,28.00){\makebox(0,0)[cc]{$(1,z_2/\varepsilon)$}}
\put(48.00,24.00){\makebox(0,0)[cc]{$(\varepsilon,z_0/\varepsilon)$}}
\put(75.00,53.00){\makebox(0,0)[cc]{$(0,z_1/\varepsilon)$}}
\put(19.00,51.00){\makebox(0,0)[cc]{$(0,z_0/z_0z_1)$}}
\put(19.00,55.00){\makebox(0,0)[cc]{$(0,z_1/z_1z_1)$}}
\put(19.00,59.00){\makebox(0,0)[cc]{$(0,z_2/z_2z_1)$}}
\put(19.00,17.00){\makebox(0,0)[cc]{$(1,z_0/z_0z_2)$}}
\put(19.00,13.00){\makebox(0,0)[cc]{$(1,z_1/z_1z_2)$}}
\put(19.00,9.00){\makebox(0,0)[cc]{$(1,z_2/z_2z_2)$}}
\put(75.00,17.00){\makebox(0,0)[cc]{$(1,z_2/\varepsilon)$}}
\put(75.00,13.00){\makebox(0,0)[cc]{$(\varepsilon,z_0/\varepsilon)$}}
\put(26.00,35.00){\vector(1,0){42.00}}
\end{picture}
}
\caption{Transition graph of the Example  \ref{KZmasodikpelda}}
\label{KZ6ka4}
\end{figure}


\noindent The corresponding transition graph can be seen in Fig.  \ref{KZ6ka4}.  Pushdown automaton  ${\mathrm V}_2$ accepts the language  $\big\{ uu^{-1}\;|\; u\in \{0,1\}^* \big\}$.  
Because  ${\mathrm V}_2$  is nemdeterministic, all the configurations obtained from the initial configuration  $(q_0,u,z_0)$ can be illustrated by a  \ki{computation tree}. For example the computation tree associated to the initial configuration  $(q_0, 1001, z_0)$ can be seen in Fig.  \ref{KZ6ka5}.  From this computation tree we can observe that, because $(q_1,\varepsilon, \varepsilon)$ is a leaf of the tree, pushdown automaton  ${\mathrm V}_2$ accepts word 1001 by empty stack. The computation tree in Fig. \ref{KZ6ka6} shows that pushdown automaton  ${\mathrm V}_2$ does not accept word  $101$, because  the configurations in leaves can not be continued and none of them has the form  $(q,\varepsilon, \varepsilon)$.

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(81.00,88.00)
\put(56.00,88.00){\makebox(0,0)[cc]{$(q_0,1001,z_0)$}}
\emline{31.00}{73.00}{1}{56.00}{85.00}{2}
\emline{56.00}{85.00}{3}{81.00}{73.00}{4}
\put(31.00,70.00){\makebox(0,0)[cc]{$(q_0,001,z_0z_2)$}}
\put(81.00,70.00){\makebox(0,0)[cc]{$(q_1,1001,\varepsilon)$}}
\emline{31.00}{67.00}{5}{31.00}{58.00}{6}
\put(31.00,54.00){\makebox(0,0)[cc]{$(q_0,01,z_0z_2z_1)$}}
\emline{6.00}{39.00}{7}{31.00}{51.00}{8}
\emline{31.00}{51.00}{9}{56.00}{39.00}{10}
\put(6.00,35.00){\makebox(0,0)[cc]{$(q_0,1,z_0z_2z_1z_1)$}}
\put(56.00,35.00){\makebox(0,0)[cc]{$(q_1,1,z_0z_2)$}}
\emline{6.00}{32.00}{11}{6.00}{23.00}{12}
\put(6.00,20.00){\makebox(0,0)[cc]{$(q_0,\varepsilon, z_0z_2z_1z_1z_2)$}}
\emline{56.00}{23.00}{13}{56.00}{32.00}{14}
\put(56.00,20.00){\makebox(0,0)[cc]{$(q_1,\varepsilon, z_0)$}}
\emline{56.00}{17.00}{15}{56.00}{8.00}{16}
\put(56.00,5.00){\makebox(0,0)[cc]{$(q_1,\varepsilon, \varepsilon)$}}
\end{picture}
\caption{Computation tree  to show acceptance of the word $1001$  (see Example \ref{KZmasodikpelda}). }
\label{KZ6ka5}
\end{figure}


\begin{figure}[t!]\centering
\unitlength=1mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(70.00,58.00)
\put(50.00,58.00){\makebox(0,0)[cc]{$(q_0, 101,z_0)$}}
\emline{30.00}{43.00}{1}{50.00}{56.00}{2}
\emline{50.00}{56.00}{3}{70.00}{43.00}{4}
\put(30.00,40.00){\makebox(0,0)[cc]{$(q_0, 01,z_0z_2)$}}
\put(70.00,40.00){\makebox(0,0)[cc]{$(q_1,101,\varepsilon)$}}
\emline{10.00}{25.00}{5}{29.00}{37.00}{6}
%\emline{29.00}{37.00}{7}{50.00}{25.00}{8}
\put(10.00,21.00){\makebox(0,0)[cc]{$(q_0,1,z_0z_2z_1)$}}
%\put(50.00,21.00){\makebox(0,0)[cc]{$(q_1,01,z_0)$}}
\emline{10.00}{18.00}{9}{10.00}{8.00}{10}
\put(10.00,5.00){\makebox(0,0)[cc]{$(q_0,\varepsilon, z_0z_2z_1z_2)$}}
\end{picture}
\caption{Computation tree to show that the pushdown automaton in Example  \ref{KZmasodikpelda} does not accept word  $101$.}
\label{KZ6ka6}
\end{figure}
\end{pelda}

\begin{tetel}
A language  $L$ is accepted  by a nondeterministic pushdown automaton ${\mathrm V}_1$ by empty stack if and only if 
it can be accepted by a nondeterministic pushdown automaton ${\mathrm V}_2$ by final state. 
\end{tetel}
\begin{biz}
\textit{a)} Let ${\mathrm V}_1=(Q,\Sigma,W,E,q_0,z_0,\emptyset)$ be the pushdown automaton which accepts 
by empty stack language $L$. Define pushdown automaton 
${\mathrm V}_2= (Q\cup \{p_0,p\},\Sigma, W\cup \{x\}, E', p_0,x, \{p\})$, 
where $p,p_0\not\in Q,\;, x\not\in W$ and

$E'= E\cup \Big\{\big(p_0, (\varepsilon, x/xz_0), q_0 \big)\Big\} \cup  
 \Big\{\big(q, (\varepsilon, x/\varepsilon), p \big)\;\big|\; q\in Q\Big\}$. 

\noindent Working of ${\mathrm V}_2$: Pushdown automaton ${\mathrm V}_2$  with an $\varepsilon$-move first goes in the initial state of ${\mathrm V}_1$, writing $z_0$  (the initial stack symbol of ${\mathrm V}_1$) in the stack (beside $x$).  After this it is working as  ${\mathrm V}_1$. If ${\mathrm V}_1$ for a given word empties its stack, then  ${\mathrm V}_2$ still has  $x$ in the stack, which can be deleted by 
 ${\mathrm V}_2$ using an $\varepsilon$-move, while a final state will be reached. ${\mathrm V}_2$ can reach a final state only if  ${\mathrm V}_1$ has emptied the stack.  

\smallskip
\textit{b)} Let ${\mathrm V}_2=(Q,\Sigma,W,E,q_0,z_0, F)$ be a pushdown automaton, which accepts language  $L$ 
by final state. Define pushdown automaton
${\mathrm V}_1=(Q\cup\{p_0,p\},\Sigma,W\cup\{x\}, E', p_0,x, \emptyset)$, where
 $p_0,p\not\in Q$, $x\not\in W$ and

$E'=E\cup  \Big\{\big(p_0, (\varepsilon, x/xz_0), q_0 \big)\Big\} 
     \cup \Big\{\big(q, (\varepsilon, z/\varepsilon), p \big)\;\big|\; q\in F, p\in Q, z\in W \Big\}\koz .$
     
   \qquad\quad\;  $\cup \; \Big\{\big(p, (\varepsilon, z/\varepsilon), p \big)\;\big|\; p\in Q,  z\in W\cup\{x\}\Big\}$


\noindent Working ${\mathrm V}_1$: Pushdown automaton ${\mathrm V}_1$  with an  $\varepsilon$-move writes in the stack beside $x$ the initial stack symbol
$z_0$ of  ${\mathrm V}_2$, then works as ${\mathrm V}_2$, i.e reaches a final state for each accepted word. After this ${\mathrm V}_1$ empties the stack by an $\varepsilon$-move.
${\mathrm V}_1$ can empty the stack only if ${\mathrm V}_2$ goes in a final state.  
\end{biz}

The next two theorems prove that the class of languages accepted by nondeterministic pushdown automata is just the set of context-free languages.

\begin{tetel}
If  $G$ is a context-free grammar, then there exists such a nondeterministic pushdown automaton ${\mathrm V}$ 
which accepts $L(G)$ by empty stack, i.e.  $L_{\varepsilon}({\mathrm V})=L(G)$.
\end{tetel}
We outline the proof only. Let $G=(N, T, P, S)$ be a context-free grammar. Define pushdown automaton
${\mathrm V}=(\{q\}, T, N\cup T, E, q, S, \emptyset)$, where  
$q\not\in N\cup T,$ and the set $E$ of transitions  is:

$\bullet$  If there is in the set of productions of $G$ a production of type $A\rightarrow \alpha$, then let put in  $E$ the transition   $\big(q, (\varepsilon, A/\alpha^{-1}), q \big)$, 
 
$\bullet$  For any letter $a\in T$ let put in  $E$ the transition $\big(q, (a, a/\varepsilon), q \big)$.
 
If there is a production  $S\rightarrow \alpha$ in $G$,  the pushdown automaton put 
in the stack the mirror of $\alpha$ with an $\varepsilon$-move. 
If the input letter coincides with that in the top of the stack, then the automaton 
deletes it from the stack. If in the top of the stack there is a nonterminal $A$, 
then the mirror of right-hand side of a production which has $A$ in its left-hand 
side will be put in the stack. If after reading all letters of the input word, 
the stack will be empty, then  the pushdown automaton recognized the input word. 

The following algorithm builds for a context-free grammar $G=(N, T, P, S)$ the pushdown automaton ${\mathrm V}=(\{q\}, T, N\cup T, E, q, S, \emptyset)$, which accepts by empty stack the language generated by $G$.

\begin{alg}{From-Cfg-to-Pushdown-Automaton($G$)}\inddef{Fromcfg@\textsc{From-Cfg-to-Pushdown-Automaton}}
1 \'  \key{for} \= all production  $A\rightarrow \alpha$\\ 
\'           \> \key{do} put in  $E$ the transition  $\big(q, (\varepsilon, A/\alpha^{-1}), q \big)$   \\ 
2 \'   \key{for} \= all terminal  $a\in T$ \\
\'           \> \key{do} put in  $E$ the transition    $\big(q, (a, a/\varepsilon), q \big)$  \\
3 \' \key{return} $V$ 
\end{alg}

If $G$ has $n$ productions and  $m$ terminals, then the number of step of the algorithm is  $\Theta(n+m)$.

\begin{pelda}\label{KZpeldax}  Let 
$\;G=(\{S,A\},\{a,b\}, \{S\rightarrow \varepsilon,\; S\rightarrow ab,\;S\rightarrow aAb,\; 
      A\rightarrow  aAb,\;A\rightarrow ab\}, S)$. Then ${\mathrm V}=(\{q\}, \{a,b\}, \{a,b,A,S\}, E, q, S, \emptyset),$
 with the following transition table.

\medskip
\begin{center}
\begin{tabular}{|c||c|c|c|c|}\hline
$\Sigma\cup\{\varepsilon\}$    & $a$ & $b$ & \multicolumn{2}{c|}{$\varepsilon$}\\  \hline
 $W$   & $a$   & $b$    & $S$ & $A$\\ \hline \hline
    & & & & \\
    & $(\varepsilon,q)$ & $(\varepsilon,q)$ & $(\varepsilon,q)$ & $(bAa,q)$ \\ 
$q$ & & & $(ba,q)$  & $(ba,q)$ \\ 
    & & & $(bAa,q)$ &\\  
    & & & & \\ \hline
\end{tabular}
\end{center}

\medskip Let us see how pushdown automaton ${\mathrm V}$  accepts word  $aabb$, 
which in grammar $G$ can be derived in the following way:
\[S\Longrightarrow aAb\Longrightarrow aabb \koz,\]
where productions  $S\rightarrow aAb$ and  $A\rightarrow ab$ were used.
Word is accepted by empty stack  (see Fig. \ref{KZlevezet}).   
\end{pelda}

\begin{figure}[t!]
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(109.00,74.00)
\put(69.00,74.00){\makebox(0,0)[cc]{$(q,aabb,S)$}}
\put(29.00,63.00){\makebox(0,0)[cc]{$(q,aabb, \varepsilon)$}}
\put(69.00,63.00){\makebox(0,0)[cc]{$(q,aabb, bAa)$}}
\put(109.00,63.00){\makebox(0,0)[cc]{$(q,aabb,ba)$}}
\put(109.00,50.00){\makebox(0,0)[cc]{$(q,abb,b)$}}
\put(69.00,50.00){\makebox(0,0)[cc]{$(q,abb,bA)$}}
\put(44.00,41.00){\makebox(0,0)[cc]{$(q,abb,bbAa)$}}
\put(94.00,41.00){\makebox(0,0)[cc]{$(q,abb,bba)$}}
\put(94.00,28.00){\makebox(0,0)[cc]{$(q,bb,bb)$}}
\put(94.00,16.00){\makebox(0,0)[cc]{$(q,b,b)$}}
\put(94.00,3.00){\makebox(0,0)[cc]{$(q,\varepsilon, \varepsilon)$}}
\put(44.00,28.00){\makebox(0,0)[cc]{$(q,bb,bbA)$}}
\put(19.00,14.00){\makebox(0,0)[cc]{$(q,bb,bbbAa)$}}
\put(64.00,14.00){\makebox(0,0)[cc]{$(q,bb,bbba)$}}
\emline{94.00}{32.00}{1}{94.00}{38.00}{2}
\emline{109.00}{53.00}{3}{109.00}{60.00}{4}
\emline{69.00}{60.00}{5}{69.00}{53.00}{6}
\emline{44.00}{38.00}{7}{44.00}{32.00}{8}
\emline{44.00}{44.00}{9}{69.00}{48.00}{10}
\emline{69.00}{48.00}{11}{94.00}{44.00}{12}
\emline{29.00}{66.00}{13}{69.00}{72.00}{14}
\emline{69.00}{72.00}{15}{109.00}{66.00}{16}
\emline{69.00}{66.00}{17}{69.00}{72.00}{18}
\emline{19.00}{17.00}{19}{44.00}{25.00}{20}
\emline{44.00}{25.00}{21}{63.00}{17.00}{22}
\emline{94.00}{7.00}{23}{94.00}{13.00}{24}
\emline{94.00}{20.00}{25}{94.00}{25.00}{26}
\end{picture} 
\caption{Recognising a word by empty stack (see Example \ref{KZpeldax}).}
\label{KZlevezet}
\end{figure}

\begin{tetel}
For a nondeterministic pushdown automaton  ${\mathrm V}$  there exists always a context-free grammar  $G$ 
 such that  ${\mathrm V}$ accepts language $L(G)$ by empty stack, i.e.  $L_{\varepsilon}({\mathrm V})=L(G)$.
\end{tetel}
Instead of a proof we will give a method to obtain grammar  $G$.
Let ${\mathrm V}=( Q,\Sigma,W,E,$ $q_0,z_0,\emptyset)$ be the nondeterministic pushdown automaton in question. 

Then $G=(N,T,P,S)$, where 

$N=\{S\}\cup \{S_{p,z,q}\;|\; p, q\in Q, z\in W\}$ and $T=\Sigma$. 

Productions in $P$ will be obtained as follows.

$\bullet$ For all state $ q$ put in  $P$ production  $S\rightarrow S_{q_0,z_0,q}$.

$\bullet$ If $\; \big(q, (a, z/z_k\ldots z_2z_1), p\big)\in E$, where 
$q\in Q$, $\; z,z_1, z_2,\ldots z_k\in W$  ($k\ge 1$) and  $a\in \Sigma\cup \{\varepsilon\}$, put in  $P$ 
   for all possible states  $p_1,p_2, \ldots, p_k$ productions 

$S_{q,z,p_k}\rightarrow aS_{p,z_1,p_1}S_{p_1,z_2,p_2}\ldots S_{p_{k-1},z_k,p_k}$.

    
$\bullet$ If $\;\big(q, (a,z/\varepsilon), p \big)\in E$, where
        $p,q\in Q, z\in W,$ and $ a\in \Sigma\cup \{\varepsilon\}$, put in  $P$ production 

$S_{q,z,p}\rightarrow a$.  

The context-free grammar defined by this is an extended one,  to which an equivalent context-free 
language can be associated.  The proof of the theorem is based on the fact 
that to every sequence of configurations, by which the pushdown automaton  
 ${\mathrm V}$ accepts a word, we can associate a derivation in grammar $G$.  
 This derivation generates just the word in question, because of productions 
 of the form $S_{q,z,p_k}\rightarrow aS_{p,z_1,p_1}S_{p_1,z_2,p_2}\ldots S_{p_{k-1},z_k,p_k}$, 
 which were defined for all possible states  $p_1,p_2, \ldots, p_k$.
In Example \ref{KZmasodikpelda} we show how can be associated a derivation 
to a sequence of configurations. The pushdown automaton defined in the example 
recognizes word 00 by the sequence of configurations 

$(q_0,00,z_0)\Longrightarrow (q_0,0,z_0z_1)\Longrightarrow (q_1,\varepsilon,z_0)\Longrightarrow (q_1,\varepsilon,\varepsilon),$

\noindent  which sequence is based on the transitions 
 
$\big(q_0, (0, z_0/z_0z_1), q_0 \big)$,

$\big(q_0, (0, z_1/\varepsilon), q_1 \big)$,

$\big(q_1, (\varepsilon,z_1/\varepsilon), q_1 \big)$.


\noindent To these transitions, by the definition of grammar $G$,  the following productions can be associated

(1) $S_{q_0,z_0,p_2} \longrightarrow 0S_{q_0,z_1,p_1}S_{p_1,z_0,p_2}$ for all states $p_1,p_2\in Q$,

(2) $S_{q_0,z_1,q_1} \longrightarrow 0$,

(3) $S_{q_1,z_0,q_1} \longrightarrow \varepsilon$. 

\noindent Furthermore, for each state  $q$ productions  $S\longrightarrow S_{q_0,z_0,q}$ were defined.

By the existence of production $S\longrightarrow S_{q_0,z_0,q}$ there exists the derivation $S\Longrightarrow S_{q_0,z_0,q}$, where  $q$ can be chosen arbitrarily.  Let choose in above production (1) state  $q$ to be equal to $p_2$. Then there exists also the derivation 

$S\Longrightarrow S_{q_0,z_0,q}\Longrightarrow 0S_{q_0,z_1,p_1}S_{p_1,z_0,q}$, 

\noindent where $p_1\in Q$ can be chosen arbitrarily. If  $p_1=q_1$, then the derivation 

$S\Longrightarrow S_{q_0,z_0,q}\Longrightarrow 0S_{q_0,z_1,q_1}S_{q_1,z_0,q}\Longrightarrow 00S_{q_1,z_0,q}$ 
 
\noindent will result.  Now let $q$ equal to  $q_1$,  then  

$S\Longrightarrow S_{q_0,z_0,q_1}\Longrightarrow 0S_{q_0,z_1,q_1}S_{q_1,z_0,q_1}\Longrightarrow 00S_{q_1,z_0,q_1}\Longrightarrow 00$, 

\noindent which proves that word 00 can be derived used the above grammar.



The next algorithm builds for a pushdown automaton  ${\mathrm V}=( Q,\Sigma,W,E,$ 
$q_0,z_0,\emptyset)$ a context-free grammar  $G=(N, T, P, S)$, which generates  
the language accepted by pushdown automaton 
 ${\mathrm V}$ by empty stack. 

\begin{alg}{From-Pushdown-Automaton-to-Cf-Grammar(${\mathrm V},G$)}
\inddef{From-Pushdown-Automaton@\textsc{From-Pushdown-Automaton-to-Cf-Grammar}}
1 \' \key{for} \= all $q\in Q$ \\
2 \'          \> \key{do} put in $P$ production $S\rightarrow S_{q_0,z_0,q}$\\ 
3 \'  \key{for} all \=  $\big(q, (a, z/z_k\ldots z_2z_1), p\big)\in E$  
  \`                \> $\rhd$    $q\in Q$, $\; z,z_1, z_2,\ldots z_k\in W$ \\  
\' ($k\ge 1$), $a\in \Sigma\cup \{\varepsilon\}$ \\
4  \'   \> \key{do} \key{for} \= all states $p_1,p_2, \ldots, p_k$  \\
5 \'         \>  \> \key{do} put in  $P$ productions  $S_{q,z,p_k}\rightarrow aS_{p,z_1,p_1}S_{p_1,z_2,p_2}\ldots S_{p_{k-1},z_k,p_k}$ \\ 
6 \'   \key{for} All $\big(q (a,z/\varepsilon), p \big)\in E$ 
    \` $\rhd$ $p,q\in Q, z\in W$, $ a\in \Sigma\cup \{\varepsilon\}$  \\  
7 \'  \> \key{do}   put in  $P$ production $S_{q,z,p}\rightarrow a$ \\ 
\end{alg}

If the automaton has $n$ states and $m$ productions, then the above algorithm executes at most $n+mn+m$ steps, so in worst case the number of steps is $O(nm)$. 
 
\medskip
Finally, without proof, we mention that the class of languages accepted by deterministic pushdown automata is a proper subset of the class of languages accepted by nondeterministic pushdown automata. This points to the fact that 
 pushdown automata behave differently as finite automata.
 
\begin{pelda}
As an example, consider pushdown automaton ${\mathrm V}$ from the Example  \ref{KZpeldax}:
${\mathrm V}=(\{q\}, \{a,b\}, \{a,b,A,S\}, E, q, S, \emptyset)$. Grammar $G$ is:

\centerline{$G=(\{S,S_a,S_b, S_S,S_A,\}, \{a,b\}, P, S) \koz ,$} 

\noindent where for all $z\in \{a,b,S,A\}$ instead of $S_{q,z,q}$ we shortly used  $S_z$. The transitions:

\begin{tabbing}

\qquad \= $\big(q, (a,a/\varepsilon ), q\big)$, \quad \qquad \=  $\big(q, (b,b/\varepsilon ), q\big) \koz ,$\\
\> $\big(q, (\varepsilon, S/\varepsilon), q\big)$, \> $\big(q, (\varepsilon,S/ba), q\big)$, \quad \qquad  
        $\big(q, (\varepsilon,S/bAa), q\big) \koz ,$\\
\> $\big(q, (\varepsilon, A/ba), q\big)$, \> $\big(q, (\varepsilon,A/bAa), q\big) \koz .$
\end{tabbing}
 
\noindent Based on these, the following productions are defined:

$S\;\, \rightarrow S_S$

$S_a\,\rightarrow a$

$S_b\,\rightarrow b$

$S_S\rightarrow \varepsilon\;|\; S_aS_b \;|\; S_aS_AS_b$

$S_A\rightarrow S_aS_AS_b\;|\; S_aS_b.$

\noindent It is easy to see that $S_S$ can be eliminated, and the productions will be: 

$S\;\, \rightarrow \varepsilon\;|\; S_aS_b\;|\; S_aS_AS_b,$

$S_A\rightarrow S_aS_AS_b\;|\; S_aS_b,$

$S_a\,\rightarrow a$, $\; S_b\rightarrow b$, \\ 
and these productions can be replaced:

$S\rightarrow \varepsilon\;|\; ab\;|\; aAb,$

$A\rightarrow aAb\;|\; ab$.
\end{pelda}

\subsection{Context-free languages}\index{language!context-free}

Consider context-free grammar $G=(N,T,P,S)$. A \ki{derivation tree} of  $G$\inddef{derivation tree} is a finite, ordered, labelled tree, which root is labelled by the the start symbol $S$, every interior vertex is labelled by a nonterminal and every leaf by a terminal.  If an interior vertex labelled by a nonterminal $A$ has $k$ descendents, then in  $P$ there exists a production $A\rightarrow a_1a_2\ldots a_k$ such that the descendents are labelled by letters   
 $a_1$, $a_2$, \ldots $a_k$.   
The \ki{result}\inddef{derivation tree!result of} of a derivation tree is a word over $T$, which can be obtained by reading the labels of the leaves from left to right. Derivation tree is also called  \ki{syntax tree}. \inddef{syntax tree}

Consider the context-free grammar $G=(\{S,A\}, \{a,b\}, \{S\rightarrow aA,\; S\rightarrow a,\; S\rightarrow \varepsilon, 
        \; A\rightarrow aA,\;A\rightarrow aAb,\;A\rightarrow ab,\;A\rightarrow b\}, S )$. It generates language  $L(G)=\{ a^nb^m\;|\;n\ge m\ge 0\}$. Derivation of word $a^4b^2\in L(G)$ is:

\centerline{$S\Longrightarrow aA \Longrightarrow aaA \Longrightarrow aaaAb\Longrightarrow aaaabb. $}

\noindent
In Fig. \ref{KZ7ka1} this derivation can be seen, which result is  $aaaabb$.

\begin{figure}\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(72.00,61.00)
\put(22.00,56.00){\circle{10.00}}
\put(7.00,45.00){\circle{10.00}}
\put(37.00,45.00){\circle{10.00}}
\put(22.00,33.00){\circle{10.00}}
\put(52.00,33.00){\circle{10.00}}
\put(37.00,20.00){\circle{10.00}}
\put(52.00,20.00){\circle{10.00}}
\put(67.00,20.00){\circle{10.00}}
\put(67.00,20.00){\circle{10.00}}
\put(45.00,7.00){\circle{10.00}}
\put(60.00,7.00){\circle{10.00}}
\put(22.00,56.00){\makebox(0,0)[cc]{$S$}}
\put(7.00,45.00){\makebox(0,0)[cc]{$a$}}
\put(37.00,45.00){\makebox(0,0)[cc]{$A$}}
\put(22.00,33.00){\makebox(0,0)[cc]{$a$}}
\put(52.00,33.00){\makebox(0,0)[cc]{$A$}}
\put(37.00,20.00){\makebox(0,0)[cc]{$a$}}
\put(52.00,20.00){\makebox(0,0)[cc]{$A$}}
\put(67.00,20.00){\makebox(0,0)[cc]{$b$}}
\put(45.00,7.00){\makebox(0,0)[cc]{$a$}}
\put(60.00,7.00){\makebox(0,0)[cc]{$b$}}
\emline{10.00}{49.00}{1}{17.50}{54.50}{2}
\emline{27.00}{54.00}{3}{34.00}{49.00}{4}
\emline{25.00}{37.00}{5}{32.50}{42.50}{6}
\emline{42.00}{43.00}{7}{49.00}{37.00}{8}
\emline{40.00}{24.00}{9}{48.00}{30.00}{10}
\emline{52.00}{28.00}{11}{52.00}{25.00}{12}
\emline{56.00}{30.00}{13}{64.00}{24.00}{14}
\emline{45.00}{12.00}{15}{49.70}{15.70}{16}
\emline{59.00}{12.00}{17}{54.30}{15.70}{18}
\end{picture}
\caption{Derivation (or syntax) tree of word $aaaabb$.}
\label{KZ7ka1}
\end{figure}


To every derivation we can associate a syntax tree. Conversely, to any syntax tree more than one derivation can be associated. For example to syntax tree in Fig. \ref{KZ7ka1}\  the derivation

\centerline{$S\Longrightarrow aA \Longrightarrow aaAb \Longrightarrow aaaAb\Longrightarrow aaaabb $}

\noindent also can be associated.


\begin{defi}
Derivation $\alpha_0\Longrightarrow \alpha_1\Longrightarrow \ldots \Longrightarrow \alpha_n$ 
is  a \ki{leftmost derivation}\inddef{leftmost derivation}, if for all 
 $i=1,2,\ldots, n-1$ there exist words $u_i\in T^*$, $\beta_i\in (N\cup T)^*$  and  productions $(A_i\rightarrow \gamma_i)\in P$, for which we have  
\smallskip \centerline{$\alpha_i=u_i A_i \beta_i \quad \textrm{and}\quad \alpha_{i+1}= u_i\gamma_i\beta_i$ \koz .} 
\end{defi}

\noindent Consider grammar:

$G=(\{S,A\}, \{a,b,c\}, \{S\rightarrow bA,\;S\rightarrow  bAS,\;S\rightarrow  a,\; A\rightarrow cS,\;A\rightarrow a\}, S )$.

\noindent In this grammar word $bcbaa$ has two different leftmost derivations:

$S\Longrightarrow bA\Longrightarrow bcS\Longrightarrow bcbAS\Longrightarrow bcbaS\Longrightarrow bcbaa$,

$S\Longrightarrow bAS\Longrightarrow bcSS\Longrightarrow bcbAS \Longrightarrow bcbaS\Longrightarrow bcbaa$.

\begin{defi}
A context-free grammar $G$ is \ki{ambiguous}\inddef{ambiguous CF grammar} if in  $L(G)$ there exists a  word with more than one leftmost derivation.  Otherwise $G$ is \ki{unambiguous.} 
\end{defi}

\medskip
The above grammar $G$ is ambiguous, because word $bcbaa$ has two different leftmost derivations.  A language can be generated by more than one grammar, and between them can exist ambiguous and unambiguous too. 
A context-free language is  \ki{inherently ambiguous,} if there is no unambiguous grammar which generates it.

\begin{pelda}
Examine the following two grammars. 

\noindent Grammar $G_1=(\{S\}, \{a,+,*\}, \{S\rightarrow S+S,\; S\rightarrow  S*S,\;S\rightarrow a\},S )$  
is ambiguous because

$S\Longrightarrow S+S\Longrightarrow a+S\Longrightarrow a+S*S\Longrightarrow a+a*S\Longrightarrow a+a*S+S
\Longrightarrow a+a*a+S $

$\Longrightarrow a+a*a+a\qquad$ and 

$S\Longrightarrow S*S \Longrightarrow S+S*S\Longrightarrow a+S*S\Longrightarrow a+a*S\Longrightarrow a+a*S+S
\Longrightarrow a+a*a +S $

$\Longrightarrow a+a*a+a.$

\noindent Grammar $G_2=(\{S,A\}, \{a,*,+\}, \{S\rightarrow A+S\;|\; A, \; A\rightarrow A*A\;|\; a\}, S)$ is unambiguous.

\noindent
Can be proved that  $L(G_1)=L(G_2)$.
\end{pelda}

\subsection{Pumping lemma for context-free languages}\index{pumping lemma!for context-free languages}
\index{context-free language!pumping lemma}

Like for regular languages there exists a pumping lemma also for context-free languages. 

\begin{tetel}[pumping lemma] For any context-free language $L$ there exists a natural number  
$n$ (which depends only on $L$), such that every word $z$ of the language longer than $n$  
can be written in the form $uvwxy$ and the following are true: 

(1)  $|w|\ge 1$,

(2) $|vx|\ge 1$,

(3) $|vwx|\le n$,

(4) $uv^iwx^iy\;$ is also in $L$ for all $i\ge 0$.
\end{tetel}

\begin{figure}[t!]\centering
\unitlength=1.00mm
\special{em:linewidth 0.4pt}
\linethickness{0.4pt}
\begin{picture}(84.00,66.00)
\put(40.00,61.00){\circle{10.00}}
\put(40.00,40.00){\circle{10.00}}
\put(40.00,21.00){\circle{10.00}}
\emline{40.00}{50.00}{1}{40.00}{49.00}{2}
\emline{40.00}{31.00}{3}{40.00}{30.00}{4}
\emline{40.00}{16.00}{5}{40.00}{12.00}{6}
\emline{40.00}{11.00}{7}{40.00}{10.00}{8}
\put(40.00,61.00){\makebox(0,0)[cc]{$S$}}
\put(40.00,40.00){\makebox(0,0)[cc]{$A$}}
\put(40.00,21.00){\makebox(0,0)[cc]{$A$}}
\put(48.00,61.00){\makebox(0,0)[cc]{$T$}}
\put(48.00,40.00){\makebox(0,0)[cc]{$T''$}}
\put(48.00,21.00){\makebox(0,0)[cc]{$T'$}}
\bezier{72}(16.00,10.00)(22.00,4.00)(29.00,10.00)
\bezier{108}(29.00,10.00)(42.00,3.00)(52.00,10.00)
\bezier{76}(52.00,10.00)(61.00,5.00)(68.00,10.00)
\bezier{76}(68.00,10.00)(77.00,5.00)(84.00,10.00)
\put(8.00,3.00){\makebox(0,0)[cc]{$u$}}
\put(22.00,3.00){\makebox(0,0)[cc]{$v$}}
\put(40.00,3.00){\makebox(0,0)[cc]{$w$}}
\put(60.00,3.00){\makebox(0,0)[cc]{$x$}}
\put(77.00,3.00){\makebox(0,0)[cc]{$y$}}
\emline{38.00}{56.00}{9}{35.00}{51.00}{10}
\emline{42.00}{56.00}{11}{44.00}{51.00}{12}
\emline{40.00}{33.00}{13}{40.00}{35.00}{14}
\emline{40.00}{52.00}{15}{40.00}{56.00}{16}
\bezier{72}(16.00,10.00)(8.00,4.00)(2.00,10.00)
\emline{2.00}{10.00}{17}{36.00}{58.00}{18}
\emline{84.00}{10.00}{19}{44.00}{58.00}{20}
\emline{16.00}{10.00}{21}{36.00}{38.00}{22}
\emline{44.00}{37.00}{23}{68.00}{10.00}{24}
\emline{52.00}{10.00}{25}{45.00}{19.00}{26}
\emline{30.00}{10.00}{27}{36.00}{19.00}{28}
\end{picture}
\caption{Decomposition of tree in the proof of pumping lemma.}
\label{KZiterlemma}
\end{figure}

\begin{biz}
Let $G=(N,T,P,S)$ be a grammar without unit productions, which generates language $L$. Let $m=|N|$ be the number of nonterminals, and let  $\ell$ be the maximum of lengths of right-hand sides of productions, i.e. $\ell=\max\big\{ |\alpha| \; |\; \exists A\in N: (A\rightarrow \alpha)\in P\big\}$.
Let  $n=\ell^{m+1}$ and $z\in L(G)$, such that $|z|> n$. Then  there exists a derivation tree $T$ with the result $z$. 
Let  $h$ be the height of $T$ (the maximum of path lengths from root to leaves). Because in $T$ all interior vertices  have at most $\ell$ descendents,  $T$ has at most  $\ell^h$ leaves, i.e.  
$|z|\le \ell^h$. On the other hand, because of $|z|> \ell^{m+1}$,  we get that $h>m+1.$ From this follows that in derivation tree $T$ there is a path from root to a leave in which there are more than  $(m+1)$ vertices. Consider such  a path. Because in $G$ the number of nonterminals is $m$ and on this path vertices different from the leaf are labelled with nonterminals, by the pigeonhole principle, it must be a nonterminal on this path which occurs at least twice.   

Let us denote by $A$ the nonterminal being the first on this path from root to the leaf which firstly repeat.
 Denote by  $T'$ the subtree, which root is this occurrence of $A$.
Similarly, denote by $T''$ the subtree, which root is the second occurrence of $A$ on this path. 
Let $w$ be the result of the tree $T'$.  Then the result of  $T''$ is in form $vwx$, while of $T$ in  $uvwxy$.  Derivation tree $T$  with this decomposition of $z$ can be seen in Fig. \ref{KZiterlemma}. We show that this decomposition of $z$ satisfies conditions (1)--(4) of lemma.
 
Because in  $P$ there are no  $\varepsilon$-productions (except maybe the case  $S\rightarrow\varepsilon$), we have  $|w|\ge 1$. Furthermore, because each interior vertex of the derivation tree has  at least two descendents (namely there are no unit productions),  also the root of $T''$ has, hence  $|vx|\ge 1$. Because  $A$  is the first repeated nonterminal on this path, the height of  $T''$ is at most  $m+1$, and from this $|vwx|\le \ell^{m+1}=n$ results.

After eliminating from  $T$ all vertices of $T''$ excepting the root, the result of obtained tree is $uAy$, i.e.
$\quad S\overset{*}{\underset{G}\Longrightarrow} uAy$.

Similarly, after eliminating $T'$ we get $\; A\overset{*}{\underset{G}\Longrightarrow} 
vAx$,\;  and finally because of the definition of $T'$ we get  $\;A\overset{*}{\underset{G}\Longrightarrow} w.\;$  
Then $ S\overset{*}{\underset{G}\Longrightarrow} uAy, 
\;\; A\overset{*}{\underset{G}\Longrightarrow}  vAx 
\;\; \textrm{and} \quad A\overset{*}{\underset{G}\Longrightarrow} w.$  
Therefore $S\overset{*}{\underset{G}\Longrightarrow} uAy\overset{*}{\underset{G}\Longrightarrow} uwy$
and $S\overset{*}{\underset{G}\Longrightarrow} uAy\overset{*}{\underset{G}\Longrightarrow} 
uvAxy\overset{*}{\underset{G}\Longrightarrow} \dots \overset{*}{\underset{G}\Longrightarrow} 
uv^iAx^iy\overset{*}{\underset{G}\Longrightarrow} uv^iwx^iy$
for all  $i\ge 1$.  Therefore, for all $i\ge 0$ we have $S\overset{*}{\underset{}\Longrightarrow} 
uv^iwx^iy$, i.e. for all  $i\ge 0$ $uv^iwx^iy\in L(G)$ .
\end{biz}

Now we present two consequences of the lemma.
\begin{kov}\label{KZnemkornyez} 
${\mathcal L}_2 \subset {\mathcal L}_1$.
\end{kov}
\begin{biz}
This consequence states that there exists a context-sensitive language which is not context-free. To prove this it is sufficient to find a context-sensitive language for which the lemma is not true. Let this language be
$ L=\{ a^mb^mc^m\;|\; m\ge 1\}.$

To show that this language  is context-sensitive it is enough to give a convenient grammar.
In Example \ref{KZoldal} both grammars are extended context-sensitive, and we know that to each extended grammar of type $i$ an equivalent grammar of the same type can be associated.

Let $n$ be the natural number associated to  $L$ by lemma, and consider the word  $z=a^nb^nc^n$. Because of   $|z|=3n>n$,
if $L$ is context-free  $z$ can be decomposed in  $z=uvwxy$ such that conditions  (1)--(4) are true. We show that this leads us to a contradiction.

Firstly, we will show that word $v$ and $x$ can contain only one type of letters. 
Indeed if  either $v$ or $x$ contain more than one type of letters, then in word $uvvwxxy$ the order of the letters  will be not the order $a,b,c$, so $uvvwxxy\not\in L(G)$, which contradicts condition  (4) of lemma.  

If both $v$ and $x$ contain at most one type of letters, then in word $uwy$ the number of different letters will be not the same, so $uwy\not\in L(G).$  This also contradicts condition  (4) in lemma. Therefore $L$ is not context-free.
\end{biz} 

\begin{kov}
The class of context-free languages is not closed under the intersection.
\end{kov}

\begin{biz} 
We give two context-free languages which intersection is not context-free. 
 Let $N=\{S,A,B\}$, $\;T=\{a,b,c\}$ and

\smallskip
\noindent $\;G_1=(N, T, P_1, S)\;$   where $P_1:$

$S\rightarrow AB$,

$A\rightarrow aAb\;|\; ab$,

$B\rightarrow cB\;|\; c,$

\smallskip
\noindent and $\;G_2=(N, T,P_2, S),\;$  where $P_2:$


$S\rightarrow AB$,

$A\rightarrow Aa\;|\; a$,

$B\rightarrow bBc\;|\; bc$.

\smallskip
\noindent Languages $L(G_1)=\{ a^nb^nc^m\;|\; n\ge 1, m\ge 1\}$ and
$L(G_2)=\{ a^nb^mc^m\;|\; n\ge 1, m\ge 1\}$  are context-free. 
But
\[ L(G_1)\cap L(G_2)= \{a^nb^nc^n\;|\; n\ge 1\}\] is not context-free (see the proof of the Consequence \ref{KZnemkornyez}). 
\end{biz}

\subsection{Normal forms of the context-free languages}
In the case of  arbitrary grammars the normal form was defined (see page \pageref{normalalak}) as grammars 
with no terminals in the left-hand side of productions. The normal form in the case of the context-free 
languages will contains some restrictions on the right-hand sides of productions. 
Two normal forms (Chomsky and Greibach) will be discussed.

\subsubsection{Chomsky normal form}\inddef{normal form!Chomsky}\inddef{Chomsky normal form}
\begin{defi} A context-free grammar $G=(N, T, P, S)$ is in Chomsky normal form, if all productions have 
form $\; A\rightarrow a\;$ or $\; A\rightarrow BC\;$, where  $A,B,C\in N$, $\;a\in T$.
\end{defi}

\begin{pelda} Grammar $G=(\{S,A,B,C\},\{a,b\}, \{S\rightarrow AB,\; S\rightarrow  CB, \; 
   C\rightarrow AS, \; A\rightarrow a, \; B\rightarrow b\}, S )$ is in Chomsky normal form and 
$L(G)= \{a^nb^n\;|\; n\ge 1\}.$
\end{pelda}

To each  $\varepsilon$-free context-free language can be associated 
an equivalent grammar is Chomsky normal form. The next algorithm transforms 
an $\varepsilon$-free context-free grammar $G=(N,T,P,S)$ in grammar 
$G'=(N',T,P',S)$ which is in Chomsky normal form. 


\begin{alg}{Chomsky-Normal-Form$(G)$}\inddef{Chomskynormal@\textsc{Chomsky-Normal-Form}}
1 \'  $N'\leftarrow N$\\
2 \'  eliminate unit productions, and let $P'$ the new set of productions \\
  \'     (see algorithm \textsc{Eliminate-Unit-Productions} on page  \pageref{KZatnevezes}) \\
3 \'  in  $P'$ replace in each production with at least two letters in right-hand side \\
  \'     all terminals $a$ by a new nonterminal $A$, and add this nonterminal to $N'$ \\
   \'  and add production  $A\rightarrow a$ to $P'$\\
4 \'  replace all productions $B\rightarrow A_1A_2\ldots A_k$, where $k\ge 3$ and   $A_1, A_2, \ldots, A_k\in N$, \\
  \'  by the following:\\ 
  \'   $\qquad B$ \quad\; \= $\rightarrow A_1C_1$,\\
  \'   $\qquad C_1$     \> $\rightarrow A_2C_2$,\\
  \'   $\qquad \ldots $ \> \\
  \'   $\qquad C_{k-3}$  \> $\rightarrow A_{k-2}C_{k-2}$,\\
  \'   $\qquad C_{k-2}$ \> $\rightarrow A_{k-1}A_k$, \\
  \'   where $C_1, C_2, \ldots , C_{k-2}$  are new nonterminals, and add them to  $N'$ \\
5 \' \key{return} $G'$
\end{alg}

\begin{pelda} Let
$G=(\{S,D\}, \{a,b,c\}, \{S\rightarrow aSc,\;S\rightarrow D, \;D\rightarrow bD, \;D\rightarrow b\}, S)$. 
It is easy to see that $\; L(G)=\{a^nb^mc^n\;|\; n\ge 0, m\ge 1\}$. Steps of transformation to Chomsky normal form are the following:

\smallskip
\noindent Step 1: $N'=\{S,D\}$

\noindent Step 2:  After eliminating the unit production $S\rightarrow D$ the productions are:

$S\rightarrow aSc\;|\; bD\;|\; b$,

$D\rightarrow bD \;|\; b$.

\noindent Step 3: We introduce three new nonterminals because of the three terminals in productions. Let these be $A,B,C$. Then the production are:

$S\rightarrow ASC\;|\; BD\;|\; b$,

$D\rightarrow BD \;|\; b$,
 
$A\rightarrow a$, 

$B\rightarrow b$,

$C\rightarrow c$.
 
\noindent Step 4: Only one new nonterminal (let this $E$) must be introduced because of a single production with three letters in the right-hand side. Therefore  $N'=\{S,A,B,C,D,E\}$, and the productions in  $P'$ are:

$S\rightarrow AE\;|\; BD\;|\; b$,

$D\rightarrow BD \;|\; b$,
 
$A\rightarrow a$, 

$B\rightarrow b$,

$C\rightarrow c$,

$E\rightarrow SC$.

\noindent All these productions are in required form. 

\end{pelda}
 
\vspace{-2mm}                                                                                                                 
\subsubsection{Greibach normal form} \inddef{normal form!Greibach}\inddef{Greibach normal form}
\vspace{-1mm}
\begin{defi}
A context-free grammar $G=(N, T, P, S)$ is in  \ki{Greibach normal form} if  all production are in the form 
$\;A\rightarrow aw$, where $\;A\in N$, $\;a\in T$, $\;w\in N^*$.
\end{defi}


\begin{pelda} Grammar $G=(\{S,B\},\{a,b\}, \{S\rightarrow aB,\; S\rightarrow  aSB, \; 
   B\rightarrow b\}, S )$ is in Greibach normal form and  $L(G)= \{a^nb^n\;|\; n\ge 1\}$.
\end{pelda}
To each  $\varepsilon$-free context-free grammar  an equivalent grammar in 
Greibach normal form can be given. We give and algorithm which transforms a context-free grammar  $G=(N,T,P,S)$ in    Chomsky normal form in a grammar $G'=(N',T, P',S)$ in Greibach normal form.

First, we give an order of the nonterminals:  $A_1, A_2, \ldots, A_n$, where  $A_1$ is the start symbol. 
The algorithm will use the notations  $x\in N'^+$, $\alpha\in TN'^*\cup N'^+$. 

\begin{algN}{Greibach-Normal-Form($G$)}\inddef{Greibachnormal@\textsc{Greibach-Normal-Form}}
1 \'  $N'\leftarrow N$ \\  
2 \'  $P'\leftarrow P$ \\
3 \'  \key{for} \= $i\leftarrow 2$ \key{to} $n$ \` $ \rhd$ Case $  A_i\rightarrow A_jx, \;j<i$ \\
4  \'            \> \key{do} \= \key{for} \= $j \leftarrow 1$ \key{to} $ i-1$     \\  
5  \'            \>          \>  \> \key{do} \= for all productions  $A_i\rightarrow A_jx$ and $A_j\rightarrow \alpha$  
 (where $\alpha$ has no $A_j$  \\
  \'            \>        \>    \> \> \quad as first letter) in  $P'$ productions $A_i\rightarrow \alpha x$,  \\
  \'            \> \> \>    \> delete from  $P'$ productions $A_i\rightarrow A_jx$\\
6  \'           \> \key{if} \= there is a production  $A_i\rightarrow A_ix$   \` $\rhd$ Case $A_i\rightarrow A_ix$ \\
7   \'            \>    \> \key{then} \= put in $N'$ the new nonterminal $B_i$,\\
   \'           \>    \>  \>  for all productions $A_i\rightarrow A_ix $ put in  $P'$ productions  $B_i\rightarrow xB_i$\\
   \'            \> \>      \>  and $B_i\rightarrow x, $delete from $P'$ production  $A_i\rightarrow A_ix $,\\      
   \'            \> \> \>  for all production $A_i\rightarrow \alpha$ (where $A_i$ is not the first letter of $\alpha$)\\
   \'            \> \> \>   put in  $P'$ production  $A_i\rightarrow \alpha B_i $    \\  
8   \'  \key{for} \= $i \leftarrow n-1$ \key{downto} $1$   \` $\rhd$ Case $A_i\rightarrow A_jx, \; j>i$\\
9   \'  \>  \key{do} \= \key{for}   $j\leftarrow i+1$ \key{to} $n$\\
10   \'   \> \> \key{do} for all  \= productions  $A_i\rightarrow A_jx$ and $A_j\rightarrow \alpha$\\
   \'    \> \> \>          put in  $P'$ production  $A_i\rightarrow \alpha x $ and\\
   \'     \> \> \>         delete from  $P'$ productions $A_i\rightarrow  A_jx $,

\algnewpageN
11   \'   \key{for} $i\leftarrow 1$ \key{to} $n $  \` $\rhd$ Case $B_i\rightarrow A_jx$\\
12   \'       \> \key {do} \key{for} \= $j \leftarrow 1$ \key{to} $n $     \\
13   \'       \>    \>  \key {do} \= for all productions  $B_i\rightarrow A_jx$ and $A_j\rightarrow \alpha$ \\
   \'    \> \> \> \quad put in  $P'$ production $B_i\rightarrow \alpha x $ and\\
   \'     \> \> \> \quad delete from $P'$ productions $B_i\rightarrow  A_jx$ \\
14 \' \key{return} $G'$ 
 \end{algN}

The algorithm first transform productions of the form $A_i\rightarrow A_jx, \;j<i$ such 
that $A_i\rightarrow A_jx, \;j\ge i$ or $A_i\rightarrow \alpha$, 
where this latter is in  Greibach normal form.
After this, introducing a new nonterminal, eliminate  
productions $ A_i\rightarrow A_ix$, and using substitutions all 
production of the form  $  A_i\rightarrow A_jx, \; j>i$ and $  B_i\rightarrow A_jx$ will be transformed in 
Greibach normal form.


\begin{pelda}
Transform productions  in Chomsky normal form 

\quad $A_1 \rightarrow A_2A_3 \;|\; A_2A_4$

\quad $A_2\rightarrow A_2A_3 \;|\; a$

\quad $A_3\rightarrow A_2A_4\;|\; b$

\quad  $A_4\rightarrow c$

\noindent in Greibach normal form.

Steps of the algorithm:

3--5:  Production $A_3\rightarrow A_2A_4$ must be transformed. For this production  $A_2\rightarrow a$ is appropriate. Put  $A_3\rightarrow aA_4$ in the set of productions and eliminate  $A_3\rightarrow A_2A_4$.

The productions will be:

\quad $A_1 \rightarrow A_2A_3 \;|\; A_2A_4$

\quad $A_2\rightarrow A_2A_3 \;|\; a$

\quad $A_3\rightarrow aA_4\;|\; b$

\quad  $A_4\rightarrow c$

6-7: Elimination of production  $A_2\rightarrow A_2A_3$ will be made using productions:

\quad $B_2\rightarrow A_3B_2$ 

\quad $B_2\rightarrow A_3$ 

\quad $A_2\rightarrow aB_2$ 
 
Then, after steps  6--7. the productions will be:

\quad $A_1 \rightarrow A_2A_3 \;|\; A_2A_4$

\quad $A_2\rightarrow aB_2 \;|\; a$

\quad $A_3\rightarrow aA_4\;|\; b$

\quad  $A_4\rightarrow c$

\quad $B_2\rightarrow A_3B_2\;|\; A_3$ 

8--10: We make substitutions in productions with $A_1$ in left-hand side. The results is:
 
 \quad $A_1 \rightarrow aA_3 \;|\; aB_2A_3\;|\; aA_4 \;|\; aB_2A_4$

11--13: Similarly with productions with  $B_2$ in left-hand side:

\quad $B_2\rightarrow aA_4B_2\;|\; aA_3A_4B_2\;|\; aA_4\;|\; aA_3A_4$ 

After the elimination in steps 8--13 of productions in which substitutions were made, the following productions, which are now in Greibach normal form, result:

 \quad $A_1 \rightarrow aA_3 \;|\; aB_2A_3\;|\; aA_4 \;|\; aB_2A_4$

\quad $A_2\rightarrow aB_2 \;|\; a$

\quad $A_3\rightarrow aA_4\;|\; b$

\quad  $A_4\rightarrow c$

\quad $B_2\rightarrow aA_4B_2\;|\; aA_3A_4B_2\;|\; aA_4\;|\; aA_3A_4$ 
 
\end{pelda}


\begin{pelda}
Language 
\[L=\big\{ a^nb^kc^{n+k}\; |\; n\ge 0, k\ge 0, n+k > 0 \big\}\]
can be generated by grammar
\[G= \big\{ \{S,R\},\{a,b,c\}, \{S\rightarrow aSc,\;S\rightarrow  ac,\;S\rightarrow  R, R\rightarrow bRc,\;R\rightarrow  bc\}, S \big\} \]

First, will eliminate the single unit production, and after this  we will give an equivalent grammar in Chomsky normal form, which will be transformed in  Greibach normal form.

Productions after the elimination of production $S\rightarrow R$:

$S\rightarrow aSc\;|\; ac\;|\; bRc\;|\; bc$

$R\rightarrow bRc\;|\; bc$.

\noindent We introduce productions  $A\rightarrow a, B\rightarrow b, C\rightarrow c$, and replace terminals by the corresponding nonterminals: 

$S\rightarrow ASC\;|\; AC \;|\;BRC   \;|\;BC, $

$R\rightarrow BRC \;|\; BC,$

$A\rightarrow a, \;B\rightarrow b, \; C\rightarrow c.$

\noindent After introducing two new nonterminals ($D$, $E$):

$S\rightarrow AD\;|\; AC \;|\;BE   \;|\;BC, $

$D\rightarrow SC,$

$E\rightarrow RC,$

$R\rightarrow BE \;|\; BC,$

$A\rightarrow a, \;B\rightarrow b, \; C\rightarrow c.$

\noindent This is now in Chomsky normal form. Replace the nonterminals to be letters $A_i$ as in the algorithm. Then,  after applying the replacements  

$S$ replaced by $A_1$,\;\;$A$ replaced by $A_2$,\;\;$B$ replaced by $A_3$,\;\;$C$ replaced by $A_4$,\;\; $D$ replaced by $A_5$, 
 
$E$ replaced by $A_6$,\;\;$R$ replaced by $A_7$,   

\noindent our grammar will have the productions:

$A_1\rightarrow A_2A_5\;|\; A_2A_4 \;|\;A_3A_6   \;|\;A_3A_4, $

$A_2\rightarrow a, \;A_3\rightarrow b, \; A_4\rightarrow c,$

$A_5\rightarrow A_1A_4,$

$A_6\rightarrow A_7A_4,$

$A_7\rightarrow A_3A_6 \;|\; A_3A_4.$

\noindent In steps  3--5 of the algorithm the new productions will occur:

$A_5\rightarrow A_2A_5A_4\;|\; A_2A_4A_4 \;|\;A_3A_6A_4   \;|\;A_3A_4A_4$ then

$A_5\rightarrow aA_5A_4\;|\; aA_4A_4 \;|\;bA_6A_4   \;|\; bA_4A_4$

$A_7\rightarrow A_3A_6 \;|\; A_3A_4, $ then

$A_7\rightarrow bA_6 \;|\; bA_4.$

\noindent Therefore

$A_1\rightarrow A_2A_5\;|\; A_2A_4 \;|\;A_3A_6   \;|\;A_3A_4, $

$A_2\rightarrow a, \;A_3\rightarrow b, \; A_4\rightarrow c,$

$A_5\rightarrow aA_5A_4\;|\; aA_4A_4 \;|\;bA_6A_4   \;|\; bA_4A_4$

$A_6\rightarrow A_7A_4,$

$A_7\rightarrow bA_6 \;|\; bA_4.$

\noindent Steps  6--7 will be skipped, because we have no left-recursive productions.  In steps 8--10 after the appropriate substitutions we have:

$A_1\rightarrow aA_5\;|\; aA_4 \;|\;bA_6   \;|\;bA_4, $

$A_2\rightarrow a$, 

$A_3\rightarrow b,$ 

$A_4\rightarrow c,$

$A_5\rightarrow aA_5A_4\;|\; aA_4A_4 \;|\;bA_6A_4   \;|\; bA_4A_4$

$A_6\rightarrow bA_6A_4 \;|\; bA_4A_4,$

$A_7\rightarrow bA_6 \;|\; bA_4.$
\end{pelda}

\vspace{-4mm}
\begin{gyak}
\ujgyak
Give pushdown automata\gyakindex{automaton!pushdown} to accept the following languages:

$L_1=\big\{a^ncb^n \;|\;n\ge 0 \big\}$, 

$L_2=\big\{a^nb^{2n}\;|\; n\ge 1  \big\}$,

$L_3=\big\{a^{2n}b^n \;|\;n\ge 0 \big\} \cup \big\{a^nb^{2n}\;|\; n\ge 0  \big\}$,
\ujgyak
Give a context-free grammar to generate language $L=\{a^nb^nc^m\;|\; n\ge 1, m\ge 1 \}$, 
and transform it in Chomsky and   Greibach normal forms. Give a pushdown automaton which accepts  $L$.


\ujgyak
What languages are generated by the following context-free grammars?

$G_1=\big(\{ S\}, \{a,b\}, \{S\rightarrow SSa, \;S \rightarrow b\}, S\big), 
\;\; G_2=\big(\{S\}, \{a,b\}, \{S\rightarrow SaS, \;S \rightarrow b\}, S\big).$
\ujgyak 
Give a context-free grammar to generate words with an equal number of  letters  $a$ and  $b$.
\ujgyak 
Prove, using the pumping lemma, that a language whose words contains an equal number of 
letters $a$, $b$ and $c$ can not be context-free.
\ujgyak 
Let the grammar $G=(V,T,P,S)$, where 

\qquad $V=\{S\}$,

\qquad $T= \{$\emph{if, then, else, a, c}$\}$,

\qquad $P=\{S\rightarrow $ \emph{if a then  S}, \; $S\rightarrow $ \emph{if a then S else S}, \; $S\rightarrow c\}$,  

\noindent Show  that word \emph{if a then if a then c else c} has two different leftmost derivations.
\ujgyak 
Prove that if $L$ is context-free, then   $L^{-1}=\{ u^{-1}\; |\; u\in L\} $  is also context-free.
\end{gyak}
\begin{fld}
\ujfld{Linear grammars}{
A grammar $G=(N,T,P,S)$ which has productions only in the form  $A\rightarrow u_1 B u_2$ 
or $A\rightarrow u$, where  $ A,B\in N, \; u, u_1, u_2 \in T^*$, 
is called a \ki{linear grammar.}\inddef{grammar!linear}   
If in a linear grammar all production are of the form  $A\rightarrow B u$ 
or $A\rightarrow v$, then it is called a left-linear grammar. \inddef{grammar!left-linear} 
Prove that the language generated by a left-linear grammar is  regular.}
\ujfld{Operator grammars}{ 
An  $\varepsilon$-free context-free grammar is called  \ki{operator grammar} \inddef{grammar!operator-} if 
in the right-hand side of productions there are no two successive nonterminals. 
Show  that, for all  $\varepsilon$-free context-free grammar an equivalent  operator grammar can be built.}
\ujfld{Complement of context-free languages}{ 
Prove that the class of context-free languages is not closed on complement.}
\end{fld}


\begin{fejmegj}
In the definition of finite automata instead of transition function we have 
used the transition graph, which in many cases help us to give simpler proofs.

There exist a lot of classical books on automata and formal languages. We mention from these the following:
two books of Aho\nevindex{Aho, Alfred V.} and Ullman \cite{Aho72,Aho73} 
in 1972 and 1973, book of Gécseg \nevindex{Gécseg, Ferenc} and 
Peák \cite{GecsegPeak}\nevindex{Peák, István (1938--1989)} in  
1972, two books of Salomaa\nevindex{Salomaa, Arto}  \cite{Salomaa69,Salomaa73} 
in 1969 and 1973, a book of Hopcroft and Ullman\nevindex{Ullman, Jeffrey David} \cite{HopcroftUl79} 
in 1979, a book of Harrison \cite{Harrison78}\nevindex{Harrison, Michael A.} in 1978,
a book of Manna \cite{Manna81}\nevindex{Manna, Zohar}, which in 1981 was published also in Hungarian. 
We notice also a book of Sipser \cite{Sipser97}\nevindex{Sipser, Michael} in 1997 and a monograph of  
Rozenberg \nevindex{Rozenberg, Grzegorz} and Salomaa \cite{Rozenberg97}\nevindex{Salomaa, Arto}. 
In a book of Lothaire\nevindex{Lothaire, M.} (common name of French authors) 
\cite{Lothaire02} on combinatorics of words we can read on other types of automata. 
Paper of Giammarresi and Montalbano \cite{Giamm99}\nevindex{Giammarresi, Dora}\nevindex{Montalbano, Rosa} 
generalise the notion of finite automata.
A new monograph is of Hopcroft,\nevindex{Hopcroft, John E.} Motwani\nevindex{Motwani, Rajeev} and 
Ullman \cite{HopcroftMoUl06}\nevindex{Ullman, Jeffrey David}. In German we recommend  the textbook  of
Asteroth\nevindex{Asteroth, Alexander} and Baier\nevindex{Baier, Christel} \cite{AsterothBaier}. 
The  concise description of the transformation in Greibach\index{Greibach normal form} 
normal form is based on this book.  

A practical introduction to formal languages is written by Webber 
\cite{Webber08}.\nevindex{Webber, Adam B.}

Other books in English:   
\cite{Brookshear89,CarrollLo89,DuKo01,Kelley95,Kozen97,Lawson04,Linz01,Linz06,Meduna00,MollArKf88,
Simon99,SimovociTe99,Straubing94,Sudkamp97}.\nevindex{Linz, Peter}\nevindex{Sudkamp, Thomas A.}  

At the end of the next chapter on compilers another books on the subject are mentioned.
\end{fejmegj}


