Tracking of objects

[ortegatron]

Hi,
I have just found your project and seems very interesting to me.
I'm working now on tracking objects, and that's how I found it.
So now I'm trying to process a lot of information: the code, the page, your posts on ycombinator news, etc.
I think I can do this departing from the part I was interested on the beginning, the tracking of objects, and then go down-up to understand other things like the minning process, economic rationale, etc. I'm not an expert on blockchain but I have some concepts.

So I have tried to find where the code relating to the tracking is, but I couldn't.
Can I take the audacy to ask you for some directions, files of code I should read first to follow this direction?
Thank you and congratulations for the project, it's a very cool and dystopian idea.

[00hello]

Hello,

I'm not sure where you're at in terms of your understanding of object tracking in computer vision but a good place to start for understanding some of the fundamentals as well as the various OpenCV trackers available is "Object Tracking Using OpenCV" by Satya Mallick.

Right now Grassland Node Lite is a very simple proof of concept that uses Centroid Tracking with a motion detector that looks for differences from frame to frame. It's ok for low power systems like the Pi. But I'm moving everything to a better tracker because it's just not going to cut it any more since it sacrifices accuracy for speed on low power systems. And tracking within the hardware confines of a Raspberry Pi just won't cut it on the main network.

This type of inaccuracy is not necessarily a bad thing for a blockchain; it's lack of precision, meaning if the way in which you're inaccurate is randomly determined then that's bad. As long as the software is inaccurate or accurate in exactly the same for exactly the same sequence of inputs for every node that computes those inputs then there will be consensus. Otherwise, you'll get what's called a fork.

Moreover, each node must be able to prove whether or not another node actually performed the computation that resulted in that output (no matter if it's accurate or inaccurate). This is important for Grassland's proof-of-work, without which no Grassland coins can be generated. The technical specification of which you'll have seen in the diagram on the main website and whose "rationale" you can read more of here in this blog post if you haven't already. That blog post starts off with a slightly different topic but delves into the economic equation i.e. "rationale" which I linked directly.

So as far as the Grassland tracking code is in this version, if you start looking through the "tracking_loop" function in multi_object_tracking.py you should see how the sequence goes from frame to frame. The motion detector starts here within "tracking_loop".

How the objects are tracked in 3D is that lnglat_homography.py infers the 3D world from the 2D world using some matrix (linear) algebra. The heart of which runs from line 126 to 152 (the end of the function).

Let me know if what I've said so far is helpful.

[ortegatron]

Thanks for the explanation. I'm currently studying the code of the duke tracker, or one of them, DeepCC. Actually I found your project googling about it, I found a post where you mention your conversation with one guy at Duke. So this eon of grassland is only doing centroid tracking, I don't understand how you multiple cameras collaborate to track an object around the sight area. Is it thought to be available on next versions? If don't really understand the output of information grasslands gives. Suppose it's working on a lot of honest nodes, which information does it offer? I read the examples of uses for stock options, but I don't really understand how's that information made out, which data does the network allow to see to participant nodes or nodes who buy tokens? Sorry if my questions are already written somewhere, I'm sure they are. But reading about it i'm finding most of the discussion centres around the possibility of attacks and not what's actually happening if we assume each node to be honest.

[00hello]

I'm currently studying the code of the duke tracker, or one of them, DeepCC. Actually I found your project googling about it, I found a post where you mention your conversation with one guy at Duke

I should mention that I later realized that the algorithm described in that blog post had exploits. It was necessary to change almost everything in order to make it robust enough to allow complete strangers to cooperate. In satisfying those constraints, the algorithm became what you read on the website.

So this eon of grassland is only doing centroid tracking

There will probably be a change to the tracker within this eon. It'll be either Mosse, KCF (OpenCV) or Dlib's tracker

I don't understand how you multiple cameras collaborate to track an object
around the sight area. Is it thought to be available on next versions?

It's always been available. A Grassland node doesn't need multiple vantage points to track the objects in a particular location. It can infer the real 3D world using a single mono (as opposed to stereo vision) camera frame. We reverse project that onto the map. You can see how that's done here in the instructions where the calibration is done.

If you have multiple video streams from multiple vantage points on a particular location, all the better. Because a node can't obviously see behind opaque objects. It can only can infer what's there based on past tracklets; like if it sees you going into the store, it can infer you're in the store. But unless there's a node in the building, it could only produce some flat probability distribution of possible locations.

If don't really understand the output of information grasslands gives.

The data nodes produce and broadcast to one another is tracklets and the information about the objects making those tracklets. This is stored and queried in raw byte format.

People don't give much regard to that type of data but if you had a photographic memory and you could remember details about people that even they'd forgotten and you were to just sit in a coffee shop next to a window in even a lightly trafficked area for about a month, you could learn a lot. At the end of that month, even strangers would be "amazed" at what you could tell them about their lives just by what you've reasonably inferred over the past few weeks.

Now take that a step further. Imagine if your "brain" had the ability to rapidly collate and cross reference even the tiniest details of everything and everyone you observed with data from wikipedia and the entire internet... wouldn't that be cool...

Suppose it's working on a lot of honest nodes, which information does it offer?

The information is exactly the same for everyone. Think of it like this. Giant steel safes, armoured trucks, security guards, tellers, passwords, external auditors, etc. etc. all ensure that the data in your local bank's microeconomic system is exactly the same for everyone. They're not there to make the data different whether or not you're honest. They're there to make it completely irrelevant whether or not you are honest. It doesn't matter. If the data in that system is dependent on someone's honesty or dishonesty then their system and the bank is broke(n)

In our case, the necessary assumption a programmer has to make when building these types of systems is that everyone is selfish and dishonest. She's making one software program for everyone, right? But no one, whether it's the programmer or a user has any guarantees everyone's going to play fair. If you get even one person who starts gaming the system then you get problems undermining those notions of Verified Computation and/or Verified Data

1. Verified Data: Over time the real world information could become less and less reliable for everyone else because the probability of it being true diminishes
2. Verified Computation: The money supply could undergo rapid inflation because someone can generate coins without the required computational difficulty and/or nodes producing valid proof-of-work not getting the coins they earned and/or the classic "double spend problem"

So that's why Grassland has perpetually increasing computational difficulty (eons with more and more powerful and detailed neural networks, FPS, frame resolution, etc. etc.), which results in

1. Grassland coins become harder to generate and therefore rarer and more valuable
2. The data becomes much more useful and reliable because of its detail, accuracy and precision
3. Node count increases because of more users attracted by 1, 4 and more coverage for 2
4. More users means means less centralization and effective attacks become harder to implement
5. Return to 1

So it's completely irrelevant whether anyone's honest or not. It's a moot point. Because encryption, constantly increasing detail/difficulty and inevitable competition make it nearly impossible for a user to not behave honestly.

I read the examples of uses for stock options, but I don't
really understand how's that information made out, which data does the
network allow to see to participant nodes...

I don't recall ever mentioning stock options with respect to the network. That would be very weird. I don't even know how that would work. Can you point me to that?

But I'm adding the networking in with the next few releases so you should see data being collected by all the nodes that are running. (That stuff would have been done sooner but I had to put that on the backburner when I realized that I would need to get funding. But now that I've taken care of that, development can start moving forward again)

And you should be able see all the tracklet data you want. Grassland is counter-intuitive, in that the more selfish, greedy and ambitious you are the more data you must, by necessity share with basically everyone. Because the useful information Grassland produces is to it a second class citizen. That information is merely the shipping container, it’s the “carrier signal” upon which the proof-of-work (without which no Grassland coins can be generated) is encoded.

... or nodes who buy tokens?

There are no tokens. Grassland is a completely self-contained system with its own cryptocurrency so it'll have its own coin. There's no other way to do it without breaking it. This should explain the difference between cryptocurrency tokens and coins.

And if by "buy" you mean buying Grassland coins on exchanges, some user will create liquidity in the market eventually I'm sure but there's nothing in code to allow them to do that yet.

Sorry if my questions are already written somewhere, I'm sure they
are. But reading about it i'm finding most of the discussion centres
around the possibility of attacks and not what's actually happening if
we assume each node to be honest.

That's the thing, we shouldn't have to assume anything in cryptography. We try to add cryptography and thermodynamics "into" the equation in such a way so that as long as the laws of physics aren't repealed, everyone has to act in a certain way because it becomes physically impossible not to; and that chances of doing so, in our case would become vanishingly small with every day that passes.

I'm not saying it's perfect, there are always vulnerabilities and bugs you have to track down. That's why they invented software versioning. But with hundreds of very polarized, vocal and sometimes vehement engineers on the internet not being able to find a flaw in the underlying theory so far, I feel confident in saying that the underlying theory is pretty sound.