Computational Linguistics Books

This book makes a good technical companion to Kahn's historic treatment in 'The Code Breakers'. It covers the technology up through the advent of computers. Its treatment is technical, going into details about how an encryption technique is performed, and how it is attacked. This book is the first place where I've seen the Enigma machine described in enough detail to understand how it works (or they worked since there were many variations and many of them are discussed here), and how to actually build (or simulate) one. It's a big book, and I carried it around for months, sometimes just diving into a chapter or topic. I loved it.

This book is the best book I have found so far on mathematical cryptology. Although the author does a fairly sketchy treatment of DES and IDEA compared to some other books out there, I feel that he makes up for it by placing all of the most common cryptographic systems in the context of a coherent and rigorous mathematical framework. Many other cryptology books fail to tie all the various cryptographic methods together using the powerful tools of modern mathematics. Dr. Bauer's text however, leaves no question in the student's mind where all the techniques fit into the theoretical framework. The second half of the book is also a pleasant surprise: a very readable but mathematically rigorous explanation of cryptanalysis. The author presents a number of statistical methods of attack that are difficult to find all in one place in the open literature. Dr. Bauer does a thorough job of explaining and augments the theory with many examples. This thorough treatment of cryptanalysis distinguishes his book from many other books on cryptology. Many authors of cryptology books pay lip-service to Kerckhoff's maxim (Only a cryptanalyst can judge the security of a crypto system.) but few bring the student enough cryptanalytic knowledge to even begin to evaluate the crypto systems presented in their books. Dr. Bauer does an excellent job of balancing cryptography with cryptanalysis. I highly recommend this book for any serious student of Cryptology. It is a real gem.

I read this book in the original German (even though reading in German is still a labor for me), and the effort was amply rewarded. This book is a first course in cryptography, at the upper undergraduate or beginning graduate level. Its competition would be books like Denning's or Beker and Piper or Koblitz' series. Denning's book is still great and worth buying (and Ms. Denning is a wonderful, accomplished, and intelligent person), but Bauer is more modern and complete. Koblitz' books are all first rate, but Bauer stays on the task of cryptology much more exactly and usefully. This is the basis of an excellent course in several German universities, especially in Munich. If I taught another course purely on cryptography (and not as part of a larger math curriculum---where Koblitz' book is best), I would certainly use this as the text. However, even though this is best, I really think everyone should still buy, read, and treasure Ms. Denning's book, Cryptology, too. (A true classic is never actually superseded.) Buy Bauer. It is better than an existing classic. While I don't have the English version yet, and cannot, therefore, vouch for the quality of the translation, I think that Springer Verlag is such a reliable editor that we can both trust that the translation will be good before we even see it.

This is an amazing book, and relatively inexpensive; Springer-Verlag has done it again.

Rather than being a dry recitation of encryption and cryptanalysis schemes, Bauer provides a great deal of information about what actually goes wrong when one tries to construct a cipher that must be used under pressure by non-cryptologists, with plenty of historical examples to illustrate his points. And he discusses at some length the ways in which cryptanalysts can hope to unravel ciphers and codes too strong to be broken by standard methods. Much of what he has to say I had never seen in print before; some of it was brand new to me. Perhaps it helps that Bauer is German, and doesn't have to write with the uneasy feeling that NSA or MI-6 is looking over his shoulder at every line he writes. For example, his explanation of how Robert Murphy compromised an American cipher in WW II so badly that the Germans could read it easily is one that I think some American officials would probably still prefer not to have in print.

Despite comments by other reviewers and by Cryptologia, I think it requires a certain mathematical sophistication to absorb much of the material in this book. The math is not hard, but Bauer implicitly assumes a mathematical mindset and a familiarity with the terminology of pure mathematics that most college undergraduates don't have. So I wouldn't choose it as the primary text for a first course in cryptology, but I would certainly use it as a supplementary text. I know of no other book that contains so much material on the practical realities of cryptology.

The book is full of very good and interesting technical information. The part on cryptanalysis is rather new and informative. The history part is mostly taken from Khan's book (you'll find some pictures in both) so there is nothing really new in this area

If you view the history of Natural Language Processing research from a certain vantagepoint, you can see a river running. The river has two banks: the procedural side, and the logical side.

Carl Hewett, living on the procedural side of the river, invented a language called "Plannner" and emphasized that knowledge consists in the ability to *do* things--to execute procedures.

Alain Colmerauer, living on the logical side of the river, invented a language called "Prolog", and emphasized that the knowledge consists of propositions which we can reason about and draw conclusions from.

On the procedural bank, Terry Winograd used Planner to create SHRDLU, a tour-de-force in Natural language processing, which showed how to make a NLP interface which could answer an impressive range of questions about blocks on a table. It could also make and execute plans involving building things with blocks. In his writings, Winograd emphasized the procedural nature of NLP understanding.

On the logical bank, Colmerauer, Rousssel, and coleages, created a French question-and-answer system which for the first time showed that every step of natural language processing, from tokenization to parsing to database query, can be performed by pure logical deduction.

Robert Kowalski was one of the first to percieve that both of these research programs were banks of the same river. As Hewett observes, Prolog reallly can be viewed as a version of Planner. The resulting vision is a stunning synthesis: Doing things can be viewed as theorem proving, and theorem proving can be viewed as doing things. There is no conflict between the proceedural and logical views--indeed they are two sides of the same coin.

Transcending these false dichotomies, these French-speaking and English-speaking researchers created what is, in the opinion of this reviewer, the mainstream of NLP research. Prolog really is the way to go if you want to do NLP research. Even hard-core lispers like Peter Norvig tacitly agree with this. Norvig wrote a book subtitled "Case studies in Common Lisp" which contains some cool NLP programs-but he didn't write those programs directly in lisp--first, he implemented prolog in lisp, and then built his NLP programs on top of prolog. Really, no matter what programming language you use, you'll eventually have to implement much of planner/prolog before you can really do natural language processing.

Through the years, there have been a series of texts which embody the forefront of this school. Michael Covington's book deserves to be mentioned in this regard. But for too long the baton has been held by Pereira and Schieber's book "Prolog and Natural-Language Analysis". Finally, at long last, the field has a book which can be considered to once again to have pushed the field forward. This book by Blackburn & Bos takes its place here at the vangard of the mainstream NLP research.

Long an underground classic--the book has been circulating samizdat in lecture note and draft form--this book was influential even before it was published. A partial list of innovations it introduces are:

1. Unquestionably the BEST discussion of quantifier scope handling techniques ever brought together between two covers. The story of quantifier scope handling is masterfully told from Montegue, through Cooper and Keller storage, to the the super-ultra-postmodern techniques of constraint-based underspecification.

2. Beautiful examples of prolog technique, including a way of modularizing prolog grammars from back-end semantic processing, which allows the authors to use the same grammar for many different processing backends, some which arn't even covered in this book.

3. Novel uses of both theorem provers _and_ model builders, to handle not just reasoning tasks, but also pragmatic tasks like assessing the informativity of sentences following one another in discourse.

Mastering the techniques presented in this book will make accessible to you a whole new vista. Further reaserch by this school, not covered in this book, but available in reasearch papers & lecture notes on the web, are presupposition handling, DRT-based discourse technique, and further advances in underspecified representations.

After reading a catichal text on prolog like "Art of Prolog", if you are ready to wade in the river of the mainstream of NLP research, this book is an excellent place to plung in.

Anyone who has had to implement any kind of NLP system that even attempts to represent semantics
will find Patrick Blackburn, and Johan Bos book refreshing and informative. So much of the material out there is either completely theoretical or the material only introduces very
introductory level examples.

Representation and Inference for Natural Language is a winner. This book presents a legitimate theoretical
introduction and well thought out examples and source code.

The experiment approach that is used in the book takes the reader through various possibilities
demonstrating their strong points and short fallings and then provides the user with
viable (real) solutions both in a theoretical fashion and in implemented source code.

Excellent job.

It has definitely helped me to implement in fairly high quality Q&A system.

Cheers to the authors!!!!!!

This volume is worth it for the opening paper, Steve Abney's brilliant "Statistical methods and linguistics". Reading this paper in Chris Manning's statistical NLP class at CMU changed the way I think about the field. I believe it should be required reading for *linguists*. In my experience, most computational linguists either don't care about mainstream theoretical linguistics, have evaluated it and dismissed it as useless, or have taken on Abney's arguments; the remainder use logic and formal language theory without statistics. I considered myself in that remainder before reading this paper.

Abney argues that Chomsky's original motive for setting up the paradigm of generative grammar though intuited grammaticality judgements was a simple expedient that allowed him to apply the mathematics of his day (automata and formal language theory). Abney dispatches "classical" generative grammar with the finesse that Chomsky showed in dismissing behaviorism. Today, information theory (read Cover and Thomas's excellent book) as applied to natural language (read Manning and Schuetze's excellent book) is the paradigm of choice for the mathematically and computationally savvy linguist.

All in all, anyone who is interested in a scientific approach to linguistics should read Abney's paper.

It is not enough to simply read great literature, we must also think about what it is we've read -- and ponder not only what the author has written, but the why and the how of the writing if we are to derive maximum value from our efforts and time spent with that particular body of work and that particular author. Such is the case with "The Body In Flannery O'Connor's Fiction: Computational Technique And Linguistic Voice" by Donald E. Hardy (Professor of English, University of Nevada - Reno). Using as the basis for his extensive analytic treatise, Professor Hardy draws upon three major essays he had written (and for the purpose of this book, re-written to make it a comprehensive and organized text) to present an innovative thesis with respect to O'Connor's use of grammatical voice and physical bodies in the texts of her novels. Combining computational and linguistic methodologies, Professor Hardy illustrates and highlights O'Connor's use of the body and its parts in her literary explorations of the sacramental and the grotesque. The result is a seminal literary analysis of impeccable interpretive scholarship. "The Body In Flannery O'Connor's Fiction" is a welcome addition to academic library reference collections in general, and an important contribution to the utilization of linguistic terminology and concepts for an interpretive approach to the humanities in particular.

A very original and well-reasoned text. I'd like to thank Dr. Sproat for his contribution to the field. This is very fine fork.

The book is well-written, easy-to-read, clear. What's that worth for a technical subject? Just about everything. First, it means you can read it like a novel, effortlessly (nearly). Second, you get a clear picture of what he is saying. So the value is there.

Yes, a lot has happened since this book was written--statistical grammars, grammar induction, constraint-based grammars, not to mention that whole web thing. No matter. This book is still a valuable stepping stone on the path to NLP nirvana, and everyone who aspires to do research in NLP should come to grips with it.

Why? Because it presents what is really the example par excellance of a unification-based grammar, the core language engine. This is really the first attempt which had any sucess at all to "scale up" nlp techniques to create a system which isn't just another toy. As such, it is one of a very very few descriptions of what it is like to build a BIG program in a logic programming language.

Is this a book which points the way to future reasarch? Well, the authors have pretty much "exhausted the search space" of unification grammars here. Both their potential and their painful limitations have been plummed, and are on display here.

But even though the time of unification grammars has passed, the lessons learned in the construction of the Core Language Engine represent, IMHO, a signature achivement, and real ground gained towards the goal.

Moreover, the spirit of this research lives on. For a bang-up-to-date discussion of where this stream of research has lead, check out Blackburn & Bos's book "Representation and Inference in Natural Language".

I hadn't heard of corpus linguistics (CL) until I chanced on a clutch of books about it in a bookstore a couple of weeks ago. After scanning a few of them, this seemed like it had the best explanation of where CL fits into the context of lingusitics. Having now read it, I'm very satisfied with my choice.

Whereas much of modern linguistics focuses on grammar, CL is focused on meaning. An outgrowth of discourse analysis, it's also entirely empirically based, relying on databases of actual language use ("corpora"), usually from printed sources. (This is in marked contrast to, say Chomskians, who rely on their innate sense as native speakers of a language to determine whether some utterance is included in that language.) Most of these databases are maintained with some foresight and care, but in a pinch you can even use Google as a CL "corpus", as the authors themselves occasionally do. They do go to some pains, though, to point out that no corpus can truly be "representative", since we have no way of capturing all the uses of a word or phrase in peoples' speech and writing. Note that unlike some other textbooks, this one doesn't teach you how to use corpora. The emphasis is on introducing the key ideas of CL, rather than on explaining its nuts and bolts. That wasn't an issue for me, but it might be for you.

The guiding insight of CL seems to be that meaning is a social phenomenon. This is in contrast to, e.g., cognitive linguistics, which holds that meanings are related to concepts that have some connection to non-verbal reality. In CL, "units of meaning" -- which may, and often are, larger than individual words -- are defined indirectly by the way people use them. These uses are influenced by our memories of the uses we have heard and read, and especially of the paraphrases and explanations that we have absorbed from such other "discourse". The authors analyze the phrase "friendly fire" as an extended example: they make the case for regarding this as a single unit of meaning, rather than as a combination of a particular sense of "friendly" with a particular sense of "fire". In other examples, they show that the benefits of CL are often most obvious in a bilingual setting. The usual format of bilingual dictionaries doesn't give you good clues as to which word is the best equivalent for the one you're trying to translate, especially when your target language isn't one in which you're native. Examination of a corpus can be much more illuminating for selecting the best translation.

By the authors' account, the benefits of CL aren't limited to translation. Claiming that "meaning is the result of a democratic process" (@134), they suggest that since CL allows us to educate ourselves about what meanings others attach to words, it can be instrumental in allowing us "to exercise our rights as free citizens in a responsible way" (id.). While I appreciate their enthusiasm for their subject, at first blush statements such as this struck me as maybe a bit grand. Also, some of their discussions of English words suffer from the fact that neither author seems to be a native speaker. E.g., I was surprised to learn that native speakers "naturally" learn that "grief" denotes bereavement in the context of guilt, and "sorrow" denotes bereavement in the context of sadness, "and not the other way around" (@82). (As if "grief" doesn't apply to many who have felt bereaved by natural disasters, the events of 9/11, the death of Princess Diana, etc.?) Despite these minor missteps, I thought this was a very stimulating and informative book, at least for a neophyte in this field.

English Corpus Linguistics: An Introduction is a must read for anyone wanting to secure a first foothold of understanding in the field. I think it would also provide a good overview for experienced corpus linguists. The book is extremely well written. There is no fluff but the text is very readable and open to anyone with an interest. The author is consistent in his prose and does a good job of balancing practical and theoretical concerns. He doesn't waste the reader's time with trifles, but makes a good entry into each well-defined subject area, provides you with a content that presents itself without considerable prerequisites of knowledge, and makes a clean exit with an apt conclusion to each chapter. The material builds on itself but is also spread out - but not to thin to loose perspective - so as not to topple the reader. I think the only thing that I might have added would have been a summary checklist with cross-reference back to individual sections. However, with 141 reading pages, it is concise enough to do a quick review by flipping through the various sections. This is important, because there is much information to take from this book as one furthers their learning.

I was waiting for this book to be published. I've been waiting for its publication since I attended a finite state morphology course at XRCE instructed by Ken Beesley. The ability of Ken to describe complex issues and processes in simple terms and to build a clear path using step by step approach using many useful examples of what you should and should not do (and why!) is exceptional! This attitude is very much reflected in the book. Reading previous academic publications of Beesley and Karttunen will make one realize that these two know their business and that they know to describe it well.

The book is intended for linguists and not for computer scientists. If you are interested to read about the theory of finite state technology, algorithms, and implementation issues then this is not the book. However, if you are a linguist planning to develop a system using finite state tools and you need an introduction to the subject, a tutorial for the xerox finite state tools and a lot of good and useful advice, this book is for you. Although this book's focus is on the Xerox finite state tools, much of the advice and tips given throughout the book is relevant to any finite state based project.

The book explains why finite state methods in general (regular languages and regular relations) and the xerox finite state tools in particular are a good choice for describing and actually building lexical transducers (which can be further extended into applications such as a morphological analyzer and generator, spell-checker, part of speech disambiguator, and more based on the same technology). It then teaches you about the syntax and usage of the Xerox finite state tools and explains a lot about various aspects of development using helpful examples, exercises and wisdom. The book has a CD containing licensed software which enables you to actually build your own systems using the Xerox finite state tools.

Although this book is a great text for any linguist using some finite state engine, it is specifically targeted at the xerox finite state tools. Note that the Xerox tools are not the only finite state tools available; Other implementations are available, notably FSA (http://odur.let.rug.nl/~vannoord/Fsa/) by Gertjan van Noord and AT&T's FSM toolset by Mohri, Pereira, Riley (http://www.research.att.com/sw/tools/fsm). This is a good time to say that the software available on CD with the book is a commercial software and the license is very restrictive and allows you to basically try it for yourself and nothing beyone that. The software itself, on the other hand, is not crippled (so it is stated) in any way. If you are interested in a software for finite state programming which has a more relaxed license than FSA util, whose license is under the GPL, is what you're looking for.

Quoting from the book: "The lesson is this: study the language and do some old-fashined pure linguistic modeling before jumping into the coding. Your programs will never be better than the linguistic model behind them." And indeed, the book does not teach you linguistics. Don't be fooled that just being a native speaker and understanding the text in the book is enough to be able to produce a useful linguistic tool using xfst and lexc. It takes a lot more than that, and the result is usually as good and robust as its foundations.

If you have some programming background note that finite state programming is very different than functional, object oriented or procedural programming. If you are familiar with Perl or unix like regular expressions, you will soon find out that the term 'regular' was severely abused on those systems, so the calculus described in the book is in many ways different: computationally weaker in some ways, more elegant in other ways. Nonetheless, linguists are bound to feel more comfortable with the Xerox tools' syntax as it was desined to suite the needs of linguists describing morphology and phonology.

Years of experience teaching how to use the tools, using the tools for various implementation of morphological analyzers, and quite a few drafts of the text which eventually turned out and materialized as this book. The organization of the text, and the terminology used makes it easy for the reader to understand new concepts. Each term is defined, explained and then examples follow. Difficult concepts are releatedly explained throughout the book. This is a fun read and I enjoyed reading it. While reading the book, whenever I felt I needed a little more explanation or another example, I just had to keep reading as the needed information was just in the next paragraph.

The first chapter is a gentle introduction to the concepts of finite state programming. The second chapter is a more systematic introduction. The exercises are useful for understanding the material. The third and fourth chapters explain xfst and lexc languages, respectively. The fifth chapter is a useful discussion about various aspects of a project and what pitfalls are expected along the way and which tools and techniques can be used to avoid or handle such pitfalls. Reading this chapter is just like reading the journal of a guru summarizing great tips. Chapter 6 on testing and debugging is yet another valuable source of information useful in many projects. The tips and lessons in chapters 5 and 6 are important to any software project, but especially using the Xerox tools, you will quickly see that the lack of a development environment and debugging tools and the flat structure of the code (allowing very limited ability to maintain your code as a set of "black box" parts with an easy interface) are a source for many problems. So following the good advice in these chapters is useful. Chapters 7 and 8 describe mechanisms implemented into the calculus which allow you to do some hacking into your code and do some "shortcuts" which allow some more expressiveness and some memory usage optimizations. Not many users will probably go ahead and use these features up front. Some may never use them at all. Chapter 9 describe useful tools and methods for automation and integration of your system as a component in a larger system.

One thing I didn't like about the book: There are many issues mentioned in brief and a footnote tells you to go to the book's web site .... to read more about it. When you actually go to the site you discover that there is no trace or evidence of that information.