Inside a Black Box

It  is an attempt to create a structure which is intermediate between a  natural and a completely digital neural network making neuron links  between neighbouring layers optical, neural cells – analog, and the  structure inside the network – parallel.

The  neural network consists of an input layer, two hidden layers, and an  output layer. The input layer is represented by 122 neurons, the second  and the third layers – by 122 and 84 neurons, and the output layer – by  50 neurons. The input layers consist only of controlled light-emitting  diodes. The neurons of the hidden layers consist of the structure  ‘phototransistor – operational amplifier – light-emitting diode’. The  links between the layers are represented by optical multipliers with  liquid-crystal light valve. The output layer is the structure  “phototransistors – analog-to-digital converter”.

The  network task is extremely simple: recognition of symbols in swarming of  fruit flies. A picture from the camera is transformed into a line of  122 different voltages for each LED of the input layer. Then, the light  passes through arrays of mutually perpendicular light conductors with  liquid-crystal light valve where light multiplication of matrix by the  weight of each link occurs. All the LEDs are linked to all the  photodetectors of the next layer via the connection weight. The  adjustment of the activation function is performed by the resistor in  the feedback circuit of operational amplifier of each neuron. It turns  on all the output LEDs of this layer which are connected to these  amplifiers, then – the light conductors with liquid-crystal matrix of  the next layer (optical multiplier), and so on.

Thus,  all the computations inside the network occur simultaneously which  approximates it to the natural one. 50 neurons of the last layer are  read sequentially and each neuron is activated to its letter or number.  The matrices of weights were defined beforehand in the process of the  network learning in a simulator where a dataset of handwritten letters  was used; the matrices don’t change during the work. On the monitor the  following can be seen: what the camera sees (fruit flies), what the  neural network sees (11×11 pixels fruit flies), and the result of work a  text consisting of a sequence of recognized letters and numbers.

This  work was carried out as part of the project #NewAnthropology for the  I.P. Pavlov Institute and the I.P. Pavlov Museum in Koltushi.

After Tomorrow. 2017-2018

If we consider the Revelation to John as a veritable scenario of the Apocalypse its realization (the reality always differs from the scenario) can look in the following way.There is a world where one part of the survived passed into a new form of life; the other part lost almost completely knowledge and technology. People who didn’t pass into the new state maintain the system of digital God-manhood taking care of bio-creatures. A part of the “not passed” worships the cult of Internet as a receptacle of everything which existed in ancient times.

The world consists of 3 parts:

1. Cargo is the cult of Internet whose followers look for the good and the revelation turning to empty screens as to portals of connection with the divine.The followers of the cult press the button and a slide projector shows them one slide of one hundred.

2. The Divine Comedy by Dante converted into electrical circuitry which in turn consists of three large parts: Hell, Purgatory, and Paradise. This is a receptacle of knowledge lost in this world in a lost language. People of cargo Internet are not able to read it but can admire it. Nevertheless the poem is the content provider for cargo Internet.The content is formed in the following way: before coming of the apocalypse we make a query in a search engine about each of one hundred parts of the poem. The response in form of screenshot is printed on a slide. Hereby, we get a set of one hundred slides which is complete volume of cargo Internet in the post-apocalyptic world.

3. The world above: its people passed to a new, post-apocalyptic stage and became the number. The Number uses bio-organisms as sources of unique data useful for search for a new Number which is ever more difficult to find. TheRevelationisintheNumber. In this case, chickens are used; their cheep is suitable to search for unique cryptokeys.In order to attract people and to maintain a stable number of workers (and bio-creature providers) the system uses a part of the number produced by the chickens to select a slide on the cult screens.

creating code installation : Sergej Mashkov

Zero City 2018

“Zero City” is an installation consisting of a number of elevator shafts of different length. Their spatial structure resembles a city where the role of each building is to move in a vertical direction. All the elevators are programmed the same way and act as a pack carrying out the general task despite the absence of a single control center. An elevator that gets incoming data from the outside (by reacting to an emerging spectator) transmits them by sending an empty platform to the floor chosen by the program. The elevator's doors open on the chosen floor. The elevator of the closest shaft receives the signal and then goes to the same floor and opens the doors across from the doors of the first elevator. Thus, we see the signal transfer as the transfer of emptiness throughout the city of the free-running elevators. A signal received in one place can move throughout the whole number of elevators or can subside completely if one of the elevator shafts gets lower than a sending link. “Zero City” is a metaphor that describes human consciousness that left its body and dispersed in the world of digital hierarchies where elevators can carry only emptiness reacting to an outer impulse or standing still in its absence.

photos of Grigory Skvortsov
photos of Grigory Skvortsov
photos of Grigory Skvortsov
photos of Grigory Skvortsov

creating code installation : Nikolaj Chetverkin

Transcendence Index 2017

The installation consists of the following 2 parts:

- a software program which analyzes texts of two periods (1900-1917 and 2000-2017) in order to calculate the index of their transcendence;

- a robotic device which cooks 2 borschts (borscht is a Russian soup based on beetroot) basing their recipes upon the data obtained after the analysis of the texts.

Stages of the Installation Functioning

1. Texts written and published from 1900 to 1917 and from 2000 to 2017 are divided into sections and then each section is correlated to one of the following borscht ingredients.

For example:

meat – philosophy;

potatoes – fiction (prose);

cabbage – texts made up by the authorities (laws, decrees, speeches);

carrots – poetry;

beetroots - news;

seasonings (onion, garlic, herbs) – private correspondence, memoirs, social networking sites;

salt – advertisement, notices (or science publications).

2. The software program finds in the texts marker words of transcendence and evaluates the transcendence index of each text and an average transcendence index in each section of the texts.

3. The program takes into consideration an average borscht recipe and calculates the coefficient of the ratio of the transcendence index of the section and the quantity of each ingredient in borscht and calculates precise content of each food item in the transcendental borscht. 4. In projections the following is shown:

- the borscht ingredients;

- the sections of the texts which correspond to them;

- the titles of the works used in current portion of borscht (in each text section);

- fragments of the analyzed texts with highlighted marker words (in conditionally real time);

- the transcendence index of the analyzed text;

- the average transcendence index of the section. The projections with the data of the texts created in 1900- 1917 and in 2000- 2017 are shown simultaneously. 

5. The screens show the calculation of the quantity of each ingredient based on the transcendence index shown in the projections. Thus, a viewer has the opportunity to understand what has been placed in to the borscht which is being cooked at the moment. 

6. The robotic device measures o ffthe necessary quantity of each ingredient and places them into soup pots with boiling water. Each ingredient is placed at its time; the degree of heating of the water is controlled by the device. The device watches over the boil-o ffof the water taking o ffand placing back the lid if necessary. When borsht is ready the heater switches off. On one side of the table there is the 1900-1917 borscht and on the other side there is the 2000-2017 borscht.

creating code installation : Sergej Mashkov, Iliya Shirshov


We translate the text of the Constitution of the Russian Federation into Morse code, and we transmit it with the help of drops of water, pauses between them, and hot irons. A drop falls only if the iron is sufciently hot. The goal of the process is to transmit information in the form of steam. The text of the Constitution is displayed on the screen so that the viewers can see the information that is being evaporated at the moment.

creating code installation : Sergej Mashkov, Nikolaj CHetverkin



Collector- a man with an electromechanical street-organ. He walks out to the streets of any city, to any place and just rotates the handle of his instrument. Yet, instead of the repetitious reproduction of the pre-inscribed melodies the street-organ processes into sound the very structure of the atmosphere of a particular site - be it a street, a waste lot, a museum, a factory, or a submarine. 

The street-organ consists of two parts - gas-chromatographer and electronic musical instrument. Rotating the handle generates electrical energy that charges the accumulator of small capacity. An air sample is collected into the sample concentrator where the admixture of air is adsorbed by the silica gel. The concentrator gets heated and rotating valve connects the concentrator to the dividing chromatography column and the sample is pushed by the bellows from the concentrator into the column chromatography. The admixture elements in the pumped air get divided based on the speed of their movement and are registered by the photo-ionization detector and other sensors. The resulted pattern of smell is used for generation of sound in the electromechanical street-organ. The device can store and play the collected patterns. This archive can be accessed through a simple screen interface. The sound is born within the instrument itself and is not created from the pre-given samples. We hear the music taken from the air here and now, it consists of the sounds created here and now. The function of the collector is to aggregate and store information that disappears momentarily after the completion of the analysis conducted within a given point in space. The collector can keep the collected patterns and carry them to other places. The collector 's activity will be video documented. As a result, we will have an archive of smells, their sounds, and the situations in which they were collected.

creating code installation : Sergej Mashkov


"A mouse vision" - the Chinese saying that means "narrow, limited view"

Animal experimentation is a common practice for people. We have used the standard metaphor of a mouse in a labyrinth to study the mechanics of how reality of shaped. The mouse is rigidly influenced by its own unrealized reality. We observe how it acts under such conditions.
While moving through the labyrinth, at each junction the mouse chooses only one of several options, the other ones remain unrealized. However, we see not only its actual route, but also all of its unrealized options in the form of virtual mice on the screen projection. The virtual mice behave independently and unpredictably. At each junction, the mouse multiplies virtual entities. When a virtual mouse moves on a collision course with the real mouse, a partition slides out between them. The virtual mice can fill almost the entire labyrinth, thus severely restricting the real mouse’s space. Being locked between partitions, the mouse stops creating its own virtual copies.

A mouse that is blocked at the crossroads cannot realize any single route.
Even if it begins to rush around in between the barriers - in the logic of self-realization - it is immobility.
At this moment the mouse stops creating its virtual copies.
Being without choice, the mouse is not without options.

In the meanwhile, active life of the virtual mice is not constrained by anything and they can move around freely.

At some point, the corridors that lead to the blocked intersection get empty, the door opens up, and the mouse can extend its life-space.

For a modern human, "to look ahead" means "to look into the future", while for the Sumerians or Babylonians looking ahead meant to see the past: the future was lying behind their backs. In Accadian the past is "days of faces, of fronts".
The word "future" is formed from the root that means "to be behind".
Events happening to the real mouse unfold in the plane of the labyrinth.

The events that take place with the virtual mice branch off the initial plane and spread around different directions. They are simultaneously past and future events for our mouse.
We cannot model the phase space of the mice, all the endless variants of their future.
Yet, if we limit the movements of the mouse mechanically, with the help of a labyrinth, it reduces the amount of possible states and allows to model the unrealized options of the future.

The barriers in the labyrinth, in a sense, serve as a prohibition of annihilation between the future and the present.

 Perhaps, the reality of the present and how we understand it is a plane that moves in a three-dimensional structure past-future, and we can investigate only those cause-effect connections that are located either close to this plane or in the past.
The events that seem strange to us, and the reasons that we do not understand may be linked to the past and present through the future.
We only have a mouse vision and see only one step ahead.

 The project was produced at the Ural Branch of the NCCA
with the help of the Center for Support and Development
of Contemporary Art ZAART, and financed by the grant
for realization of innovative projects in the sphere of contemporary art
of the Federal focus-program "Russian Culture (2012-2018)",
Ministry of Culture, Russian Federation.

Svetlana Usoltseva
Kseniya Zhideleva

creating code installation 144MLN, NEXUS

Where Dogs Run
2013 - 2014

фото: Роман Мокров

Мы используем привычную метафору мыши в лабиринте для изучения механизмов формирования реальности. Мышь  подвергается жесткому воздействию своей собственной нереализованной реальности.  Мы наблюдаем, как именно,  она строит свою действительность в этих условиях.
Мышь, проходя по лабиринту , на каждом перекрестке выбирает лишь один из вариантов, все остальные остаются нереализованными. Мы видим не только реализованный путь, но и упущенные возможности в виде виртуальных мышей на проекции. Виртуальные мыши действуют самостоятельно и непредсказуемо.   На каждом перекрестке  мышь плодит виртуальные сущности. В момент, когда виртуальная мышь направляется на встречу реальной, - в лабиринте между ними  выдвигается стенка. Виртуальные мыши могут заполнить почти весь лабиринт и ограничить  пространство реальной мыши до предельного минимума. Мышь, запертая между стенками, перестает создавать свои виртуальные копии. Не имея выбора -она не теряет возможностей.
Таким образом, нереализованные возможности мыши, формируют траекторию и объем ее жизненного пространства. События, происходящие с реальной мышью,  разворачиваются в плоскости лабиринта. События, происходящие с виртуальными мышами,  разворачиваются во всех направлениях. Они одновременно и прошедшие и будущие события для нашей мыши.
 Простая  структура лабиринта сокращает количество  возможных состояний мыши, что позволяет нам моделировать нереализованные варианты ее будущего и влиять на ее настоящее.












The project was shown at the exhibition “Scientific Museum in the XXIst Century” in the Ural Branch of NCCA from February 20 to March 16, 2013; from March 22 to April 12 in Samara; also at the “Zero Gravity” exhibition from September 10 to October 15, 2013 (curated by Elena Selina), which was a part of the special project of V Moscow Biennale of Contemporary Art. “Trialogue” project brought the artists an award of Dmitry Zimin’s “Dynasty” fund.

Three white balls in the field of snow. The balls are robots capable of autonomous and coordinated movements. They are exchanging information about their reciprocal positions and are using as a motion equation the law which is qualitatively analogous to the gravity law. The balls are attracted and repulsed depending on the space between them and their mass. They are moving not as point masses ready to turn in any moment, but as vectored points. This is a mathematical model of three-body system. It is impossible to predict their trajectories over the long period of time. To a purely mathematical model we are adding its physical embodiment; as a result, we have a multitude of not accounted for and non-foreseeable parameters, such as field irregularities, patches of clinging snow, mechanical control imprecisions etc. These new parameters make it impossible to predict the robots’ movements even one minute ahead.
We can only watch developments and sudden changes in relationships between them.
Through video, we are capturing the process of incessant calculations which is chaotically changing mutual trajectories of the balls. Having created the system, we cannot control it. We can only serve
as physical inaccuracies influencing the current mathematical operation; at the same time we don’t know how exactly we are influencing these calculations. We renounce control and accept the impossibility of control. This is an autistic system, close-looped to endlessly calculate itself.

created on the notes depicted at "The Garden of Earthly Delights" by
Hieronymus Bosch
played by "Vox Vulgaris" (disk "The shape of medieval music to come", 2003)

Project was realized with the support of Dynasty Foundation (Moscow) and the Ural Branch of National Center of Contemporary Art

Svetlana Usoltseva

Denis Perevalov

Scientific Advisers
Viktor Orlov,
Saint-Petersburg State University, physics
Vladimir Surdin,
The Sternberg Astronomical Institute (Moscow), physics

Computer simulation of three-body problem
Eugene Butikov,
Saint-Petersburg State University, physics

Voice Over
Petr Vinokurov

Thanks to

Roman Plotnikov, Marina Sokolovskaya, Alisa Prudnikova, Ksenia Zhidelyova, Daria Parkhomenko, Stanislav Stankovskiy, Artem Korzukhin

Where dogs run. 2013

Три белых шара в снежном поле. Шары – роботы, способные двигаться автономно и согласованно. Они обмениваются информацией о взаимном положении в пространстве и  используют в качестве уравнения движения закон, качественно аналогичный закону гравитации. Шары притягиваются и отталкиваются в зависимости от расстояния между ними и массы их тел. Они движутся не как материальные точки, готовые повернуть в любой момент, а как точки с направлением. Это математическая модель системы 3-х тел. Невозможно предсказать их траектории на длительном промежутке времени. К чистой математической модели мы добавляем физическое ее воплощение и возникает масса не учитываемых и  не прогнозируемых параметров, такие как неровности поля, налипший местами снег, неточности механического управления и т.д. Эти новые параметры делают невозможным предсказание поведение роботов даже на минуту вперед.
Мы можем только наблюдать за развитием и внезапными изменениями отношений между ними.
Мы фиксируем на видео процесс непрерывных вычислений, хаотично меняющих взаимные траектории шаров. Создав систему, мы не можем управлять ее. Мы можем только выступать в роли физических погрешностей, влияющих на текущую математическую операцию, при этом мы не можем знать, как именно мы влияем на эти вычисления. Мы отказываемся от контроля и мы признаем невозможность контроля. Эта аутичная система, замкнутая на постоянное вычисление самой себя.

в проекте использована композиция,
созданная на основе нот, изображённых на триптихе
Иеронима Босха «Сад радостей земных»
Ансамбль "Vox Vulgaris"
Юн Кулльблад (ударные, колёсная лира)

Проект реализован при поддержке: фонда Дмитрия Зимина "Династия"
Уральского филиала Государственного центра современного искусства
Куратор: Светлана Усольцева

Денис Перевалов

Научные  консультанты:

Виктор  Орлов
СПбГУ профессор, доктор физико-математических наук
Владимир Сурдин
кандидат физико-математических наук
доцент ГАИШ имени Штернберга МГУ
Компьютерное моделирование задачи многих тел:
Евгений Бутиков
СПбГУ профессор кафедры общей физики

текст читает Пётр Винокуров

Благодарим за помошь в реализации проекта:

Романа Плотникова
Марину Соколовскую
Алису Прудникову
Ксению Жиделеву
Дарью Пархоменко
Станислава Станьковского
Артёма Корзухина

Куда бегут собаки
2013 год