What do the numbers in NEAR Condition signify in ruta script? - uima

I used the below script to find list items;
Script:
SuspectedList{NEAR(SuspectedList,1,5)->MARK(DZP_List)};
SuspectedList{-PARTOF(DZP_List),NEAR(SuspectedList,1,5,false)->MARK(DZP_List)};
SuspectedList contains headings and list items, out of which I'm trying to extract only list items using NEAR condition and mark them as DZP_List.
Input:
<para113>43. The Cyrus Legend (ca. 580 BCE)</para113>
<para114>[Insert image ML38 with caption: A fifteenth-century French manuscript depicts the dream of Astyages, in which a great vine grew from the womb of Mandane, the mother of Cyrus.]</para114>
<para115>After the destruction of the Assyrian Empire at the end of the seventh century BCE by the Medes and Chaldeans, the Medes established a powerful kingdom that extended from Iran to central Anatolia. At this time the Persians, even though they had kings of their own, were vassals of the Medes. In the mid sixth century, the Persian Cyrus, a descendent of the Persian royal family, led a revolt against the Medes and founded the Persian Empire. As was the case with many important persons of antiquity, legends grew up about Cyrus’ origins, such as this tale related in the mid fifth century BCE by the Greek historian Herodotus. In this case, there was a common folk tale motif of the noble child raised in disguise, much as in the legends of Romulus and Remus (Reading 10.1 below) and King Arthur. </para115>
<para130>44. The Behistun Inscription (ca. 520 BCE)</para130>
<para131>[Insert image ML39 with caption: The Behistun inscription, carved into a cliffside on Mount Behistun near Kermanshah in western Iran, describes the rise to power of the Persian king Darius I.]</para131>
<para132>In order to get the news out about important events during their reigns, Persian kings had huge rock reliefs carved onto the sides of cliffs. For Darius I (522–486 BCE), the most important event of his reign was the very first, his proclamation as Great King, which occurred under somewhat irregular circumstances. Although Darius was, or at least claimed to be, a direct descendent of the first Persian king Achaemenes, he was only a distant cousin of Cyrus, the founder of the Persian Empire. It therefore was crucially important that he establish his legitimacy, and one of the means he employed to do this was the creation of this rock carving that told the story of his rise to power. It not only depicted him fully enthroned as king but also had an accompanying inscription, written in Persian, Babylonian, and Elamite. The account contained in a massive inscription carved into the side of Mount Behistun in Iran is very candid about the problems that Darius had establishing his control over the empire, and, in typical near eastern fashion, attributes his legitimacy to the support of a god, in this case Ahura Mazda, the primary god of the Zoroastrian religion. It provides in excruciating detail accounts, with exact places and dates, of the battles that Darius fought in his quest to secure the throne.</para132>
<para133>Source: L.W. King, R.C. Thompson, The Sculptures and Inscription of Darius the Great on the Rock of Behistûn in Persia (London, 1907).</para133>
<para135>(1) I am Darius, the Great King, King of Kings, the King of Persia, the king of countries, the son of Hystaspes, the grandson of Arsames, the Achaemenid.</para135>
<para136>(2) King Darius says: My father is Hystaspes; the father of Hystaspes was Arsames; the father of Arsames was Ariaramnes; the father of Ariaramnes was Teispes; the father of Teispes was Achaemenes. </para136>
<para137>(3) King Darius says: That is why we are called Achaemenids; from antiquity we have been noble; from antiquity has our dynasty been royal.</para137>
<para138>(4) King Darius says: Eight of my dynasty were kings before me; I am the ninth. Nine in succession we have been kings.</para138>
<para139>(5) King Darius says: By the grace of Ahura Mazda am I king; Ahura Mazda has granted me the kingdom.</para139>
<para140>(6) King Darius says: These are the countries that are subject unto me, and by the grace of Ahura Mazda I became king of them: Persia, Elam, Babylonia, Assyria, Arabia, Egypt, the countries by the Sea, Lydia, the Greeks, Media, Armenia, Cappadocia, Parthia, Drangiana, Aria, Chorasmia, Bactria, Sogdia, Gandara, Scythia, Sattagydia, Arachosia and Maka; twenty-three lands in all.</para140>
<para141>(7) King Darius says: These are the countries that are subject to me; by the grace of Ahura Mazda they became subject to me; they brought tribute unto me. Whatsoever commands have been laid on them by me, by night or by day, have been performed by them.</para141>
<para142>(8) King Darius says: Within these lands, whosoever was a friend, him have I surely protected; whosoever was hostile, him have I utterly destroyed. By the grace of Ahura Mazda these lands have conformed to my decrees; as it was commanded unto them by me, so was it done.</para142>
<para143>(9) King Darius says: Ahura Mazda has granted unto me this empire. Ahura Mazda brought me help, until I gained this empire; by the grace of Ahura Mazda do I hold this empire.</para143>
<para144>(10) King Darius says: The following is what was done by me after I became king. A son of Cyrus, named Cambyses, one of our dynasty, was king here before me. That Cambyses had a brother, Smerdis by name, of the same mother and the same father as Cambyses. Afterward, Cambyses slew this Smerdis. When Cambyses slew Smerdis, it was not known unto the people that Smerdis was slain. Thereupon Cambyses went to Egypt. When Cambyses had departed into Egypt, the people became hostile, and the lie multiplied in the land, even in Persia and Media, and in the other provinces.</para144>
<para145>(11) King Darius says: Afterward, there was a certain man, a Magian, Gaumâta by name, who raised a rebellion in Paishiyâuvâdâ, in a mountain called Arakadriš. On the fourteenth day of the month Viyaxana did he rebel. He lied to the people, saying: “I am Smerdis, the son of Cyrus, the brother of Cambyses.’ Then were all the people in revolt, and from Cambyses they went over unto him, both Persia and Media, and the other provinces. He seized the kingdom; on the ninth day of the month Garmapada he seized the kingdom. Afterward, Cambyses died of natural causes. </para145>
<para146>(12) King Darius says: The kingdom of which Gaumâta, the Magian, dispossessed Cambyses, had always belonged to our dynasty. After that Gaumâta, the Magian, had dispossessed Cambyses of Persia and Media, and of the other provinces, he did according to his will. He became king. </para146>
<para147> (13) King Darius says: There was no man, either Persian or Mede or of our own dynasty, who took the kingdom from Gaumâta, the Magian. The people feared him exceedingly, for he slew many who had known the real Smerdis. For this reason did he slay them, “that they may not know that I am not Smerdis, the son of Cyrus.’ There was none who dared to act against Gaumâta, the Magian, until I came. Then I prayed to Ahura Mazda; Ahura Mazda brought me help. On the tenth day of the month Bâgayâdiš I, with a few men, slew that Gaumâta, the Magian, and the chief men who were his followers. At the stronghold called Sikayauvatiš, in the district called Nisaia in Media, I slew him; I dispossessed him of the kingdom. By the grace of Ahura Mazda I became king; Ahura Mazda granted me the kingdom.</para147>
<para148>(14) King Darius says: The kingdom that had been wrested from our line I brought back and I reestablished it on its foundation. The temples that Gaumâta, the Magian, had destroyed, I restored to the people, and the pasture lands, and the herds and the dwelling places, and the houses that Gaumâta, the Magian, had taken away. I settled the people in their place, the people of Persia, and Media, and the other provinces. I restored that which had been taken away, as it was in the days of old. This did I by the grace of Ahura Mazda, I labored until I had established our dynasty in its place, as in the days of old; I labored, by the grace of Ahura Mazda, so that Gaumâta, the Magian, did not dispossess our house.</para148>
<para149>(15) King Darius says: This was what I did after I became king.</para149>
SuspectedList:
43. The Cyrus Legend (ca. 580 BCE)
44. The Behistun Inscription (ca. 520 BCE)
(1) I am Darius, the Great King, King of Kings, the King of Persia, the king of countries, the son of Hystaspes, the grandson of Arsames, the Achaemenid.
(2) King Darius says: My father is Hystaspes; the father of Hystaspes was Arsames; the father of Arsames was Ariaramnes; the father of Ariaramnes was Teispes; the father of Teispes was Achaemene.
(3) King Darius says: That is why we are called Achaemenids; from antiquity we have been noble; from antiquity has our dynasty been royal.
(4) King Darius says: Eight of my dynasty were kings before me; I am the ninth. Nine in succession we have been kings.
(5) King Darius says: By the grace of Ahura Mazda am I king; Ahura Mazda has granted me the kingdom.
(6) King Darius says: These are the countries that are subject unto me, and by the grace of Ahura Mazda I became king of them: Persia, Elam, Babylonia, Assyria, Arabia, Egypt, the countries by the Sea, Lydia, the Greeks, Media, Armenia, Cappadocia, Parthia, Drangiana, Aria, Chorasmia, Bactria, Sogdia, Gandara, Scythia, Sattagydia, Arachosia and Maka; twenty-three lands in all.
(7) King Darius says: These are the countries that are subject to me; by the grace of Ahura Mazda they became subject to me; they brought tribute unto me. Whatsoever commands have been laid on them by me, by night or by day, have been performed by them.
(8) King Darius says: Within these lands, whosoever was a friend, him have I surely protected; whosoever was hostile, him have I utterly destroyed. By the grace of Ahura Mazda these lands have conformed to my decrees; as it was commanded unto them by me, so was it done.
(9) King Darius says: Ahura Mazda has granted unto me this empire. Ahura Mazda brought me help, until I gained this empire; by the grace of Ahura Mazda do I hold this empire.
(10) King Darius says: The following is what was done by me after I became king. A son of Cyrus, named Cambyses, one of our dynasty, was king here before me. That Cambyses had a brother, Smerdis by name, of the same mother and the same father as Cambyses. Afterward, Cambyses slew this Smerdis. When Cambyses slew Smerdis, it was not known unto the people that Smerdis was slain. Thereupon Cambyses went to Egypt. When Cambyses had departed into Egypt, the people became hostile, and the lie multiplied in the land, even in Persia and Media, and in the other provinces.
(11) King Darius says: Afterward, there was a certain man, a Magian, Gaumâta by name, who raised a rebellion in Paishiyâuvâdâ, in a mountain called Arakadriš. On the fourteenth day of the month Viyaxana did he rebel. He lied to the people, saying: “I am Smerdis, the son of Cyrus, the brother of Cambyses.’ Then were all the people in revolt, and from Cambyses they went over unto him, both Persia and Media, and the other provinces. He seized the kingdom; on the ninth day of the month Garmapada he seized the kingdom. Afterward, Cambyses died of natural causes.
(12) King Darius says: The kingdom of which Gaumâta, the Magian, dispossessed Cambyses, had always belonged to our dynasty. After that Gaumâta, the Magian, had dispossessed Cambyses of Persia and Media, and of the other provinces, he did according to his will. He became king.
(13) King Darius says: There was no man, either Persian or Mede or of our own dynasty, who took the kingdom from Gaumâta, the Magian. The people feared him exceedingly, for he slew many who had known the real Smerdis. For this reason did he slay them, “that they may not know that I am not Smerdis, the son of Cyrus.’ There was none who dared to act against Gaumâta, the Magian, until I came. Then I prayed to Ahura Mazda; Ahura Mazda brought me help. On the tenth day of the month Bâgayâdiš I, with a few men, slew that Gaumâta, the Magian, and the chief men who were his followers. At the stronghold called Sikayauvatiš, in the district called Nisaia in Media, I slew him; I dispossessed him of the kingdom. By the grace of Ahura Mazda I became king; Ahura Mazda granted me the kingdom.
(14) King Darius says: The kingdom that had been wrested from our line I brought back and I reestablished it on its foundation. The temples that Gaumâta, the Magian, had destroyed, I restored to the people, and the pasture lands, and the herds and the dwelling places, and the houses that Gaumâta, the Magian, had taken away. I settled the people in their place, the people of Persia, and Media, and the other provinces. I restored that which had been taken away, as it was in the days of old. This did I by the grace of Ahura Mazda, I labored until I had established our dynasty in its place, as in the days of old; I labored, by the grace of Ahura Mazda, so that Gaumâta, the Magian, did not dispossess our house.
(15) King Darius says: This was what I did after I became king.
DZP_List should contain all the above except the first two entries (43 & 44). They both are headings (followed by paragraphs).
But my output excludes 43, 44 and also 12, where 11 ends with a space. If the space is removed, I get the correct output. 2 also ends with a space, but is working fine.
Also I tried changing the interval in the NEAR condition with various values like;
0,1 - includes 43 & 44 (heading) also
1,5 - 12 is missing, but when the space in 11 (in input file) is removed, it works fine.
1,10 - works perfect !!
and get weird results.

The numbers specify the min and max amount of annotations of the type RutaBasic in the given direction in which the given type needs to occur. RutaBasic are the minimal annotations available and are automatically created. Normally, there is one for each TokenSeed annotation like CW, NUM, PERIOD...

Related

Lines that have three upper case characters in parenthesis like (LTI) - remember to escape certain characters

This is a homework for postgresql regular expression that is to write a regular expression meeting the following requirement: Lines that have three upper case characters in parenthesis like (LTI) - remember to escape certain characters. I wrote the expression: purpose, ([A-Z]{3}). the answer is supposed to be 3 rows, but I got only 2 rows with my query. I don't why and solution. the purpose as followed:
we are dedicated to animal welfare and preventing cruelty to stray, abandoned or abused dogs and cats in Harris and Montgomery counties of texas. We will focus our efforts on Trap Neuter and Return(TNR) of cats and helping cats and dogs in need. We have established a sanctuary for animals that can't be adopted due to medical condition or age. These activities will be conducted by board members, the volunteers and members of the organization. We will provide shelter, water, food and medical care and rehabilitation for animals in need. Medical care will be performed by local veterinarians and will include but not limited to sterilization, vaccinations, testing for disease, as well any other medical care recommended by the Vet to promote the health of the animal. We focus on animals in need in Harris and Montgomery county Texas. We perform TNR which is common practice in cat rescue. We trap stray animals and spay/neuter/vaccinate and vetted and return them to their location if possible. We also sponsor free cat and dog sterilazation service to residents of the counties. We also offer a voucher system for residents in need who can't afford sterlization services. We also devote our efforts to animals in the 5th ward of Houston which is a low income area. We also offer residents of this area with pet food, vetting and pet care as needed. Our first and foremost goal is to educate and assist area residents who may need help understanding how to better care for their pets and who may need economic assitance to ensure their pets receive essential medical care. We provide residents with vouchers that cover the complete cost of having animals altered and vaccinated against rabies and common diseases. In addition we provide vaccinations and worming in the field free of charge. We also assist residents in placing unwanted puppies born in the area and work with those who own the many chained dogs we observe to provide shelter to protect them from the elements.137186
the National Coalition for History (NCH) advocates on federal legislative and regulatory issues affecting historians, archivists, teachers, researchers, and other stakeholders. Since 1982, the National Coalition for History (formerly the National Coordinating Committee for the Promotion of History) has served as the voice for the historical and archival professions in Washington, DC. The history coalition is a non-profit organization organized under Section 501(c)(3) of the Internal Revenue Code. NCH is solely supported by contributions from its member organizations and the general public.
operation of a commercial paper program for the benefit of california state university (CSU) to enhance the capital financing for projects, equipment, and facilities serving csu.
I am wondering what's wrong with my expression and the solution. thanks
Your expression ([A-Z]{3}) matches 3 uppercase letters and captures them as a group, ie ABC would be matched from "abc ABC abc" (with or without brackets).
To match literal brackets you must escape them:
\([A-Z]{3}\)

When running my Kafka consumer in local, it is only consuming half of the message

This is the topic created -
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mango
and a very long message is produced -
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mango
>The wolves stopped in their tracks, sizing up the mother and her cubs. It had been over a week since their last meal and they were getting desperate. The cubs would make a good meal, but there were high risks taking on the mother Grizzly. A decision had to be made and the wrong choice could signal the end of the pack.Balloons are pretty and come in different colors, different shapes, different sizes, and they can even adjust sizes as needed. But don't make them too big or they might just pop, and then bye-bye balloon. It'll be gone and lost for the rest of mankind. They can serve a variety of purposes, from decorating to water balloon wars. You just have to use your head to think a little bit about what to do with them.According to the caption on the bronze marker placed by the Multnomah Chapter of the Daughters of the American Revolution on May 12, 1939, “College Hall (is) the oldest building in continuous use for Educational purposes west of the Rocky Mountains. Here were educated men and women who have won recognition throughout the world in all the learned professions.”He heard the loud impact before he ever saw the result. It had been so loud that it had actually made him jump back in his seat. As soon as he recovered from the surprise, he saw the crack in the windshield. It seemed to be an analogy of the current condition of his life.Hopes and dreams were dashed that day. It should have been expected, but it still came as a shock. The warning signs had been ignored in favor of the possibility, however remote, that it could actually happen. That possibility had grown from hope to an undeniable belief it must be destiny. That was until it wasn't and the hopes and dreams came crashing down.Sitting in the sun, away from everyone who had done him harm in the past, he quietly listened to those who roamed by. He felt at peace in the moment, hoping it would last, but knowing the reprieve would soon come to an end. He closed his eyes, the sun beating down on face and he smiled. He smiled for the first time in as long as he could remember.Out of another, I get a lovely view of the bay and a little private wharf belonging to the estate. There is a beautiful shaded lane that runs down there from the house. I always fancy I see people walking in these numerous paths and arbors, but John has cautioned me not to give way to fancy in the least. He says that with my imaginative power and habit of story-making a nervous weakness like mine is sure to lead to all manner of excited fancies and that I ought to use my will and good sense to check the tendency. So I try.His parents continued to question him. He didn't know what to say to them since they refused to believe the truth. He explained again and again, and they dismissed his explanation as a figment of his imagination. There was no way that grandpa, who had been dead for five years, could have told him where the treasure had been hidden. Of course, it didn't help that grandpa was roaring with laughter in the chair next to him as he tried to explain once again how he'd found it.He watched as the young man tried to impress everyone in the room with his intelligence. There was no doubt that he was smart. The fact that he was more intelligent than anyone else in the room could have been easily deduced, but nobody was really paying any attention due to the fact that it was also obvious that the young man only cared about his intelligence.Greg understood that this situation would make Michael terribly uncomfortable. Michael simply had no idea what was about to come and even though Greg could prevent it from happening, he opted to let it happen. It was quite ironic, really. It was something Greg had said he would never wish upon anyone a million times, yet here he was knowingly letting it happen to one of his best friends. He rationalized that it would ultimately make Michael a better person and that no matter how uncomfortable, everyone should experience racism at least once in their lifetime.Sometimes that's just the way it has to be. Sure, there were probably other options, but he didn't let them enter his mind. It was done and that was that. It was just the way it had to be.The computer wouldn't start. She banged on the side and tried again. Nothing. She lifted it up and dropped it to the table. Still nothing. She banged her closed fist against the top. It was at this moment she saw the irony of trying to fix the machine with violence.What were they eating? It didn't taste like anything she had ever eaten before and although she was famished, she didn't dare ask. She knew the answer would be one she didn't want to hear.Do you really listen when you are talking with someone? I have a friend who listens in an unforgiving way. She actually takes every word you say as being something important and when you have a friend that listens like that, words take on a whole new meaning.The amber droplet hung from the branch, reaching fullness and ready to drop. It waited. While many of the other droplets were satisfied to form as big as they could and release, this droplet had other plans. It wanted to be part of history. It wanted to be remembered long after all the other droplets had dissolved into history. So it waited for the perfect specimen to fly by to trap and capture that it hoped would eventually be discovered hundreds of years in the future.She wondered if the note had reached him. She scolded herself for not handing it to him in person. She trusted her friend, but so much could happen. She waited impatiently for word.Stranded. Yes, she was now the first person ever to land on Venus, but that was of little consequence. Her name would be read by millions in school as the first to land here, but that celebrity would never actually be seen by her. She looked at the control panel and knew there was nothing that would ever get it back into working order. She was the first and it was not clear this would also be her last.She looked at her little girl who was about to become a teen. She tried to think back to when the girl had been younger but failed to pinpoint the exact moment when she had become a little too big to pick up and carry. It hit her all at once. She was no longer a little girl and she stood there speechless with fear, sadness, and pride all running through her at the same time.She was in a hurry. Not the standard hurry when you're in a rush to get someplace, but a frantic hurry. The type of hurry where a few seconds could mean life or death. She raced down the road ignoring speed limits and weaving between cars. She was only a few minutes away when traffic came to a dead standstill on the road ahead.She had been told time and time again that the most important steps were the first and the last. It was something that she carried within her in everything she did, but then he showed up and disrupted everything. He told her that she had it wrong. The first step wasn't the most important. The last step wasn't the most important. It was the next step that was the most important.
But only half of them is consumed -
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mango --from-beginning
>The wolves stopped in their tracks, sizing up the mother and her cubs. It had been over a week since their last meal and they were getting desperate. The cubs would make a good meal, but there were high risks taking on the mother Grizzly. A decision had to be made and the wrong choice could signal the end of the pack.Balloons are pretty and come in different colors, different shapes, different sizes, and they can even adjust sizes as needed. But don't make them too big or they might just pop, and then bye-bye balloon. It'll be gone and lost for the rest of mankind. They can serve a variety of purposes, from decorating to water balloon wars. You just have to use your head to think a little bit about what to do with them.According to the caption on the bronze marker placed by the Multnomah Chapter of the Daughters of the American Revolution on May 12, 1939, “College Hall (is) the oldest building in continuous use for Educational purposes west of the Rocky Mountains. Here were educated men and women who have won recognition throughout the world in all the learned professions.”He heard the loud impact before he ever saw the result. It had been so loud that it had actually made him jump back in his seat. As soon as he recovered from the surprise, he saw the crack in the windshield. It seemed to be an analogy of the current condition of his life.Hopes and dreams were dashed that day. It should have been expected, but it still came as a shock. The warning signs had been ignored in favor of the possibility, however remote, that it could actually happen. That possibility had grown from hope to an undeniable belief it must be destiny. That was until it wasn't and the hopes and dreams came crashing down.Sitting in the sun, away from everyone who had done him harm in the past, he quietly listened to those who roamed by. He felt at peace in the moment, hoping it would last, but knowing the reprieve would soon come to an end. He closed his eyes, the sun beating down on face and he smiled. He smiled for the first time in as long as he could remember.Out of another, I get a lovely view of the bay and a little private wharf belonging to the estate. There is a beautiful shaded lane that runs down there from the house. I always fancy I see people walking in these numerous paths and arbors, but John has cautioned me not to give way to fancy in the least. He says that with my imaginative power and habit of story-making a nervous weakness like mine is sure to lead to all manner of excited fancies and that I ought to use my will and good sense to check the tendency. So I try.His parents continued to question him. He didn't know what to say to them since they refused to believe the truth. He explained again and again, and they dismissed his explanation as a figment of his imagination. There was no way that grandpa, who had been dead for five years, could have told him where the treasure had been hidden. Of course, it didn't help that grandpa was roaring with laughter in the chair next to him as he tried to explain once again how he'd found it.He watched as the young man tried to impress everyone in the room with his intelligence. There was no doubt that he was smart. The fact that he was more intelligent than anyone else in the room could have been easily deduced, but nobody was really paying any attention due to the fact that it was also obvious that the young man only cared about his intelligence.Greg understood that this situation would make Michael terribly uncomfortable. Michael simply had no idea what was about to come and even though Greg could prevent it from happening, he opted to let it happen. It was quite ironic, really. It was something Greg had said he would never wish upon anyone a million times, yet here he was knowingly letting it happen to one of his best friends. He rationalized that it would ultimately make Michael a better person and that no matter how uncomfortable, everyone should experience racism at least once in their lifetime.Sometimes that's just the way it has to be. Sure, there were probably other options, but
Two things may happen: if you are pasting the message in a terminal it can get truncated, so the producer is actually sending only a part of it.
Another possibility is that you need to set offset.metadata.max.bytes in your broker to an higher value (default 4096 bytes).
To check if is the console truncating you can pass the data via file, ie:
cat yourMsg.txt | kafka-console-producer.sh --broker-list localhost:9092 --topic mango
In general I would check if there are configuration matches with the length of the message.
For example your magic numbers are the length of the string received by the consumer that seems to be around 4092 (maybe more with the carriage return),
so 4092-4096 (4096 is the closest power of 2).
Not sure with the console one but usually the text is UTF-8 that is 4 bytes per character 4096 * 4 = 16368 bytes. With that numbers I would search what may make sense with the configuration.
The default size of a message produced in Kafka is 1 MB. If the message is greater than the size you can incur data loss.
To increase the size of your message, set the following configuration: message.max.bytes and allocate the size of message you want to produce and consume.
Also, set the following configuration max.partition.fetch.bytes in your consumer to fetch the size of messages you want to receive.
You can also read more about these configurations in this link:
https://kafka.apache.org/documentation/#brokerconfigs

tools for visualising money flows in a complex company structure

I'm working on a project where we want to visualise money flows in complex company structures. The context is real-estate investment in Berlin, Germany, which faces currently an acute shortage in apartments.
Berlin is a highly attractive market for investors, as the market is still very affordable compared to other European capitals. But as authorities do not support building affordable housing, and don't build themselves, private investors have a relative dominance.
Among private investors, a big part use company networks spread across Europe to aggressively avoid taxes. They use every legal possibility, with mother companies European tax havens like Jersey, Luxemburg and Cyprus. The finance minister of Berlin estimates losses due to tax avoidance by RE investors to 100 to 200 Mio € yearly.
Our project aims to show how these investors work. The goal is to make this highly technical subject understandable for non-specialists, first step to raise public awareness.
We want to show in detail how money flows across company structures that can have 30+ subsidiaires spread across Europe. Here are examples: https://ibb.co/album/kPT6Jv
Example of a complex company structure: the investor Taliesin, Berlin/Jersey
You can see that as an organigram, these structures are more like a puzzle and need a specialist to explain what they mean.
The representation should be dynamic, so that you can see the rents rising from the houses, trickling through the network, ending at the beneficiairies. Ideally, you'd see how much money remains at each station of the network.
A user, or a tenant, could enter the amount of his rent in a field, say 1000 €, and then follow its path through the companies. At the end he'd see that of his rent, 140 € remain in the house for upkeeping, 220 € are used for management costs of the owners, 240 € go to the shell companies and the managers companies, 380 € go the banks as interest rates on the company's loans, 20 € are paid in taxes.
This example is taken from the analysis of the investor Taliesin in Berlin, one of the most agressive tax avoiders, and are actual figures. I co-wrote an article on Taliesin for the Berlin daily Tagesspiegel, 8 oct 2016: https://www.tagesspiegel.de/berlin/share-deals-auf-dem-berliner-immobilienmarkt-wie-investoren-den-kreuzberger-buechertisch-ausbooteten/14658204.html
In our project, "Who do I actually pay rent to", we'll instruct tenants how to research and transmit the data to an analyst-team, who'll then verify and reconstruct the money flow.
Hope this question is not too long.
So back to the actual question:
is there a tool to visualise this type of flows? I've found a couple of blockchain/bitcoin visualisation and ready-mades, like Key lines
do you know developers who could help in finding and adapting the visualisation tools? There will be a couple of other visualisations to do.
The project will be funded and there will be a large chunk of the money reserved to the visualisation tool, as this is main aspect.
Any questions, please ask.
Looking forward for your answers.
Best,
Adrian
Is there a tool to visualise this type of flows? I've found a couple
of blockchain/bitcoin visualisation and ready-mades, like Key lines.
You need a data visualisation app.
In short Yes, this can be built with Python and the use of d3.js to display the data in the way you want.
Python : https://www.python.org/
D3.JS : https://d3js.org/
do you know developers who could help in finding and adapting the
visualisation tools? There will be a couple of other visualisations to
do
Yes, me.

How to cluster preferences of customers?

Briefly, i have data set of sales for the year. I need to cluster users preferences. I have vectors these sales for each user. For example,
i have vector with purchases of John this consist of 1 or zero(bought this thing or not).
[1 ,0 ,0 ,1 ,0 ]
[product1,product2, product3,product4,product5]
it means that John bought product1 and product4. May be someone saw good articles abouts this. I need to cluster and offer other customers different things, for example if some customers have the nearest neighbor(with enough small distance) then i will able to offer him things from this neighbor.
in advance, sorry for my bad english and thanks
I am interested in ideas or articles!
size of real matrix of these customers is 10^8*10^6
Use market basket analysis, not cluster analysis.
In particular, it does not assume that every customer is typical (part of a cluster) or that a customer can only be part of one.
Rules of the kind
butter, bread -> marmelade
are excellent for product offering.

Aggregate Root Choice (for all Football/Soccer fans)

I have been reading StackOverflow for weeks, but I still could not decide whether my DDD Aggregate Root choice is correct. Long story short -- here are the entities. It is about the football/soccer domain:
League, Team and Match
Each Team can participate in one or more Leagues by playing Matches (i.e. English Premier League, UEFA Champions League). Each team has HomeMatches and AwayMatches in a certain League. Each Match has a League, HomeTeam and AwayTeam.
Each League has many Matches.
I think I need to have two repositories -- LeagueRepository where I can get all the matches for a certain league for a certain period. Through this repository I will automatically update the database when a round of matches has been played and I will record the results accordingly.
I also need a TeamRepository, where I can get all the Matches for a certain team in different leagues for different periods of time. This is for statistical purposes, i.e. give me all Liverpool Home Matches in the English Premier League for the last 10 years. Yeah, you guessed right -- it is about betting chances and odds calculations :)
Long story short -- my Domain is the Football/Soccer World. Those of you who follow the sport know those details and what a League, Team and Match is.
Is it OK to have two separate aggregate roots -- League and Team. I can reach a given match through either one of them. Is this OK with the DDD design?
Or should I introduce a new Entity called Sport and make it the sole aggregate root. Then a Sport will have many Leagues and many Teams.
I am using EF code-first approach and I am trying to identify my Repositories and aggregate roots. If you were designing this database, how would you structure those three entities -- League, Team and Match. Of course we are over-simplifying things here.
All your thoughts and comments will be greatly appreciated.
Thanks in advance.
To me, Sport seems like a waste of a concept. If you don't need to model, soccer, baseball, basketball, etc then I imagine the sport model would be mostly empty and a waste of space. If you think of your program in terms of teams and leauges, then that is the two repositories I would stick with. What advantage to a Sport repository do you see other then just having one root?
Three white papers that you should read to better understand how you should go about choosing your aggregates: http://dddcommunity.org/library/vernon_2011/