RNC News

The Russian National Corpus celebrates its 20th anniversary!

On April 29, 2004, the RNC website was opened for free access. But the creation of the RNC began much earlier, back in 2000. It is symbolic that the official "birthday" of the Corpus is on April 29 – the birthday of the Russian linguist, author of the Grammatical Dictionary of the Russian language A. A. Zaliznyak (1935-2017).

It all started with the idea of creating a complete collection of texts that would be culturally representative and reflect the diversity of prose written between 1965 and 2000. Currently, the RNC consists of 49 corpora with a total volume of more than two billion tokens. For 20 years, the Corpus has become an indispensable tool for linguists, teachers, students and anyone interested in the Russian language.

Congratulations to the creators of the RNC and those who help it develop! Thanks to you, the Corpus continues to grow and improve, providing new opportunities for learning the Russian language.

For those who are interested in learning more about the history and modern capabilities of the Corpus, we have prepared a set of materials:

• Explore how the corpus looked 20 years ago, in the RNC Museum. 
• Immerse yourself in the history of the creation and development of the RNC in a special project of the “Bolshoy gorod”.  
• Read the User Guide and learn how to use the corpus for different tasks.
• Explore the publications about the RNC in the recently updated section. We recommend paying attention to the recent publication on the fundamental reconstruction and modernization of the RNC platform.
• Download and apply neural network models, which are used to mark up words and texts of the Corpus, for your own tasks 
• Find out how to get the offline version of the Corpus for research.

Those who want to participate in the development of the corpus are invited to join the group «‎Друзья НейроКРЯ»‎. You will be the first to learn about upcoming projects and will be able to participate in them. Recently, we launched a new experiment to find out which definitions of words are better perceived by users: taken from dictionaries or generated by a neural network.

A new section is now available on the Corpus website. It describes the RNC neural network models used for annotating words and texts within the Corpus.

The users have access to the following tool:

• the tokenizer
• vector space models searching word associates and customized for 7 domains
• models for morphemic annotation
• models for annotating genre, topic, and type of text

The new section will be useful for everyone who is interested in natural language processing and wants to learn more about what machine learning technologies are used in RNC. Users can consult descriptions of the models or download them for their own use. Before downloading a model please read the license agreement and accept its terms.

In April, the Old East Slavic corpus was considerably upgraded. It now features new types of search results, such as Frequency, Statistics, and n-grams. Using the Frequency feature users can build frequency lists of tokens and constructions. For example, one can check which nouns are coordinated most often in the corpus of the Early Medieval texts (‘Boris and Gleb’, ‘fear and trembling’ and others). The query results can be sorted by context. Frequency dictionaries are available while customizing subcorpus, and they can be compared to the lexical frequencies of the whole corpus.

The arrival of new functionality expands the possibilities of using the corpus and automates routine processes that previously took considerable time.

We continue to roll out new functionality already available in the advanced corpora, such as Main, Media, and Learning, to other corpora. An improved version of the “From 2 to 15” corpus is now available to users of the RNC. All the texts within the corpus feature resolved grammatical homonymy and syntactic annotation. Syntactic relations search and collocation search are now available, as well as new output types such as frequency, n-grams, statistics.

The Word at a Glance function has been updated, and new types of sorting by context have been added.
In the Word at a Glance you can see that the words мама 'mom' and папа 'dad' are used much more often in texts for the children of 7-8 years old, and the words бабушка 'grandma' and дедушка 'grandpa' has an equal frequency rating for both the children of 7-8 years and for teenagers of 14-15 years.

The bar next to the fragment indicating the age of readers who should understand these fragments is now clickable. When you click, you will see the calculated classical readability indices: Flesch-Kincaid Index, Coleman-Liau Index, Automatic Readability Index, Simple Measure of Gobbledygook, Dale-Chull readability formula. 

In anticipation of the 20th anniversary of the National Corpus, we have significantly updated the publications page on our website. The list of publications about the Corpus has been expanded: the number of publications has increased by about 5 times! The section now includes both academic articles and other types of publications such as interviews, instructions, and social media posts.

The page of publications about the Corpus has advanced functionality: now you can find a publication about the Russian National Corpus in the search bar or using the filters on the right.

By default, the most popular filters are shown to the user. To see all available filters on the publications page, click "Show all". Combining multiple filters narrows the search and allows publications to be selected using multiple criteria.

Some publications can be downloaded by clicking on the icon to the right of the title. Other publications open in a separate window. You can share the list of selected publications by clicking on the "Copy link" button.

Two new parallel corpora are available. The Japanese-Russian language pair has more than 400 thousand tokens and includes fiction texts and news translated from Japanese. The Khakas-Russian texts prepared for the RNC on the basis of the Electronic Corpus of the Khakas Language feature more than 1 million tokens and cover folklore (including 19th century records), written fiction, and journalism.

The existing parallel corpora have also been expanded. The Portuguese pair (now 1.6 million tokens) and the Czech pair (4.3 million tokens) have grown the most.

New widgets are now available at the Word at a glance in the National Media, Educational and Russian classics corpora.

Sketches, Word frequency and Similar words have appeared in the National Media and the Russian classics corpora. Since Word at a glance is built on the base of the corpus texts, sketches and similar words for the same word are different in different corpora. For example, in the texts of the National media corpora, шутка ('joke') is most often злой ('evil') and первоапрельский ('April Fools’') and in the works of Russian classics — it is колкий ('sharp') and забавный ('funny').

The Statistics widget in all three corpora has been updated. Follow the link to find out in which type of texts of Russian classics the word anecdote is more often used.

In March, the Syntactic сorpus was significantly upgraded. New information and search fields appeared within SynTagRus. Starting from Word 2, one may specify coreference and temporal relation. In the Additional Features field one can now search for elided words, i.e. words that are omitted in a sentence but are present in its syntactic structure.

Searching by microsyntactic annotation allows finding stable expressions of different types. For example, by selecting the constructions vsë ravno 1 (~'all the same'), vsë ravno 2 and vsë ravno 3, the user can see the specific semantic features of this polysemous idiom ('in any way', 'indifferently' and 'equivalently', respectively) and the specific features of its usage (for example, vsë ravno 2 and 3 act as predicate, while vsë ravno 1 does not).

Sorting the search results by text date, author's date of birth, annotation date and random sorting is now available. By default, the results are sorted by the annotation date of the texts.

Hints appeared in the menus of syntactic relations, lexical functions and morphological features. Clicking on the (?) button in the neighboring window will open the corresponding description in the User's Guide.

The Russian Classics corpus has implemented automatic annotation using neural network mechanisms. It now offers the same search and statistical tools as the Main, Media and other "advanced" corpora: frequency, n-grams, statistics, sketches in the Word at a glance function, search by syntactic relations, comparison of subcorpora by frequency dictionaries, etc. The Russian Classics is the only one of the RNC corpora where both verse and written prose are represented, and it is now possible to select poetic or prosaic texts separately. With all this we can compare what the Russian classics wrote about. For example, we can problematize alleged "human-centeredness" of Russian classical literature (see picture), and also notice that poets (see Zhukovsky, Baratynsky, or Lermontov mentioned duša 'soul' much more often than authors of prose (see Radishchev, Gogol, or Turgenev).

The Syntactic Corpus now offers the possibility to select a subcorpus by basic parameters, such as author, text title, date of creation and author's year of birth, as well as by genres and text types and by markup date.

Follow our news on the website and social networks, in March we will continue to improve the Syntactic Corpus!