Top trends

387 views
Skip to first unread message

Miguel Freitas

unread,
Feb 15, 2014, 8:24:36 PM2/15/14
to twist...@googlegroups.com
Hi folks,

I've committed an experimental top trending hashtag code to twister-core and twister-html.

It seems to work, it is nice, helps finding about what is going on and everything... but it's very naive implementation so i guess it will take no time until someone tries to abuse it...

i'm already thinking in improving it by incorporating the notion of "degrees of separation". We all know we are friend of everybody in the world with less than 6 degrees of separation, right? :-)


I thought of having a crawler thread in twister-core which would periodically go recursively into the following lists (starting with local users) until a certain deep of separation (2? 3? 4?). Then it would collect the list of these usernames as "trusted" users.

The thing with top trends would be that only posts from users in this list would be considered. This way, fake users or spammers (who are not followed) would not be able to send thousands of posts just to make a top trend. Their posts would be simply ignored.

What do you think? How many degrees should we use?

regards,

Miguel

Shift rus

unread,
Feb 16, 2014, 10:25:11 AM2/16/14
to twist...@googlegroups.com
one remark, i tested that with @jpfox and if you have closed ports you don't get trends


воскресенье, 16 февраля 2014 г., 5:24:36 UTC+4 пользователь Miguel Freitas написал:

Klaus Alexander Seistrup

unread,
Feb 18, 2014, 12:50:00 AM2/18/14
to twist...@googlegroups.com
Would it be possible to implement ”Top trends” as “Trending” instead?  I.e., based on relative rather than absolute numbers, so that the tags that have seen the largest increase in ocurrance during the past, say 12 or 24 hours, will appear on the trending list.

For instance, @soltempore post 60-70 twists each day with the tag #sunrise, but since the number is constant I wouldn't call it “trending”.  In the same way, @blockhash posts a large number of twists with the tag #blockchain, but this number is also relatively constant and shouldn't be countet as trending.

(Using relative numbers, the trending field could even display the bottom trenders, i.e., those with the fastest decreasing relative trend.  I've always felt it was unfair to only display “most viewed”, “top favourites”, etc.).

Cheers.

bah...@gmail.com

unread,
Feb 18, 2014, 11:28:08 AM2/18/14
to twist...@googlegroups.com
On Tuesday, February 18, 2014 6:50:00 AM UTC+1, Klaus Alexander Seistrup wrote:
Would it be possible to implement ”Top trends” as “Trending” instead?  I.e., based on relative rather than absolute numbers, so that the tags that have seen the largest increase in ocurrance during the past, say 12 or 24 hours, will appear on the trending list.

You probably mean to compute 'trending' as (an approximation of) the the first derivative of tags occurrences in some given interval;

then maybe 'popping trends' as second derivative...

(forgive my pedantry that just popped out..)

  bahque

Klaus Alexander Seistrup

unread,
Feb 18, 2014, 12:08:03 PM2/18/14
to twist...@googlegroups.com
@bahque

First derivative, yes.  Second derivative (acceleration?) never occured to me…

bah...@gmail.com

unread,
Feb 18, 2014, 12:58:47 PM2/18/14
to twist...@googlegroups.com
On Tuesday, February 18, 2014 6:08:03 PM UTC+1, Klaus Alexander Seistrup wrote:
@bahque

First derivative, yes.  Second derivative (acceleration?) never occured to me…

caveat, idle speculations follow:

in a given interval, the most frequent hashtags could not mean they are highly interesting, maybe just the result of obstinacy or spamming

a higher first derivative should mean the interest in the tag is growing faster (accelerating)

a higher second derivative could mean not much, but maybe the topic is going to burst popular - who knows

  bahque


Miguel Freitas

unread,
Feb 18, 2014, 1:52:05 PM2/18/14
to twist...@googlegroups.com
Yes, these are all possible ideas. We can even make the algorithm selectable so each user may decide whats fits better for him.

Just bear in mind we have access to just a small subset of all the posts at any given time. It's hard to guess if this small subset would provide enough data for more elaborate algorithms but i think it is worth trying.

regards,

Miguel

Shift rus

unread,
Feb 19, 2014, 1:42:30 PM2/19/14
to twist...@googlegroups.com
dont forget about local trends based on GEO ip / location info in profile

вторник, 18 февраля 2014 г., 22:52:05 UTC+4 пользователь Miguel Freitas написал:

Сёма Мрачный

unread,
Apr 23, 2014, 8:36:21 AM4/23/14
to twist...@googlegroups.com
We can even make the algorithm selectable so each user may decide whats fits better for him.
 personally I want to have in client 1 configurable list of top trends (with possibility to delete/forbid some hashtags) and 1 one list of favorite hashtags.

Miguel Freitas

unread,
Apr 27, 2014, 10:11:05 PM4/27/14
to twist...@googlegroups.com
On Wed, Apr 23, 2014 at 9:36 AM, Сёма Мрачный <scarylit...@gmail.com> wrote:
We can even make the algorithm selectable so each user may decide whats fits better for him.
 personally I want to have in client 1 configurable list of top trends (with possibility to delete/forbid some hashtags) and 1 one list of favorite hashtags.

hey, that's a nice idea, thanks! i should make a new RPC command to forbid some hashtags from being accounted within the daemon. then we add some UI element (what about a [X] besides the hashtag?) to use it.

i guess publishing the list of deleted hashtags may also be of some use later (for example: in future, we may rely on our friend's list of deleted hashtags etc)

regards,

Miguel

Shift rus

unread,
Apr 28, 2014, 12:42:24 PM4/28/14
to twist...@googlegroups.com
my proposal about spam protection fits bad hashtag protection https://github.com/miguelfreitas/twister-core/issues/157

понедельник, 28 апреля 2014 г., 6:11:05 UTC+4 пользователь Miguel Freitas написал:
Message has been deleted
Message has been deleted

Denis Ryabov

unread,
Sep 13, 2014, 7:18:04 PM9/13/14
to twist...@googlegroups.com
In this letter I would like to start discussion of an idea to add "data" torrent in additional to existing "messages" torrent (with all user's posts). There may be a lot of ways to use this torrent, and first of all we can implement files attaching (mainly, images, but it might be videos, files, long URLs for a built-in URL shortener, long posts beyond 140 chars limit, etc.). We started to discuss it in Russian Twister community few months ago, and Miguel Freitas supports this idea as well.

Current things about implementation of this idea:

1. Twister stores user's messages as separate pieces of the torrent file, and we could use the same idea for additional "data" torrent (that is, 1 files = 1 piece), but it's better to split large file into several pieces to speed up loading from several peers. Usually, piece size of about 256Kb is used in large torrent, so we can use this value as well (note that piece size limits maximal amount of data that may be stored in the torrent). So, each reference to a file in "data" torrent should contain file size and starting piece index in the torrent (amount of pieces to load is easily derived from known file size). Each piece of data torrent should contain data block followed by PGP signature to be sure it's valid content (the same way is used currently in Twister's messages torrent).

2. Main torrent should support new type of messages (in addition to posts, RTs, and DMs): message (description) + file information. Additional type allows to easily filter messages and display all "file" messages separately (like Twitter does in photos/videos tab). This type of messages should include following info:

- description message (standard 140 chars limit)
- start piece number in data torrent
- file size
- original file name (optionally)
- mime type (optionally)

3. Full data torrent may be quite big in size (e.g. if it contains videos), so this torrents shouldn't be joined into swarm to minimize total size of data stored on user's nodes, and followers are the only users who distribute the torrent. Most likely there should be a way to set maximal file size to load/share automatically, and bigger files can be loaded by direct request only (e.g. by click on "load" button in client's interface). In additional, we can use some time-to-live technique and remove loaded files (pieces of the torrent file) if they are not requested for certain time (but continue to store on author's Twister instance(s)), or to don't load all of very old files (older than time-to-live) by a new follower. There is a chance that old files will be "forgotten" if data torrent of author's Twister instance is damaged, but maybe it's a reasonable compromise between functionality and required storage. Note that Miguel's opinion is "The problem is availability: the data might still be there, but if the author is offline then for all practical effects the data is lost. That's the reason i don't believe in torrents requesting data only by-demand, requesting earlier we increase data availability for the whole network. Too low availability will kill this cool concept imho.", so any feedback and ideas are welcome.

4. Note that there is a way to send files in direct messages. The idea is to encrypt file using a symmetric algorithm with random key and include this key in the direct message.

Any response will be greatly appreciated.

-------
Best regards,
Denis Ryabov (twister: @denis)

Julian Steinwachs

unread,
Sep 16, 2014, 5:14:12 AM9/16/14
to twist...@googlegroups.com
Hey,

My 2 cents: Torrents are not metadata secure. Distribution of the torrent creates direct connections between the author and the readers of that post. I'm not saying that torrents are a bad idea, but they dial down privacy significantly. They should definitely be optional.

Greetings,
Tschaul
--
You received this message because you are subscribed to the Google Groups "twister-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to twister-dev...@googlegroups.com.
To post to this group, send email to twist...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Miguel Freitas

unread,
Sep 21, 2014, 8:14:31 AM9/21/14
to twist...@googlegroups.com
Hi Denis,

Thanks a lot for the thoughts on this issue and for concisely expressing the new proposal. This is a great idea!

One thing I'd like to comment is the new "contract" that is embodied into this proposal. I mean the rule to decide where data is stored/duplicated in twister network.

Currently, twister "contract" is that nodes should join the effort in storing posts from other users and do so by getting randomly assigned to keep some posts on DHT entries or to join some user's torrents. Given the size limits on posts I believe this is a fair agreement: one knows that his HD space won't be abused by others but he must still contribute a little bit. In exchange he gets the ability to go offline and have his posts still available somewhere else, hosted by other users.

With image or arbitrary file storage the game changes: space occupied on one's HD may rise fast due to some abuser, or one's may try to store contents on your computer you disapprove (i don't want to contribute my own bandwidth and HD sharing any beheading video, for example).

So what I've suggested Denis is that this new "media" torrent doesn't get to be governed by the same rules as current "posts" torrent does. That is: users would never get randomly assigned to join anyone's new "media" torrent to help sharing the contents, unless he agrees with that.

My proposal is a simple concept: if you follow some user, then you join his "media" torrent and helps sharing the files. If you don't follow, you don't host/share any of his data.

Additional limitations should be implemented, as Denis noted. That may include time-to-expire, maximum file or size etc. 

regards,

Miguel

PS, implementation detail: since tracker doesn't automatically joins the media torrent, we should rely on the same standard mechanism of bittorrent trackers. that is: swarm members should periodically notify the tracker through DHT of their own IP/port.

Miguel Freitas

unread,
Sep 21, 2014, 8:18:56 AM9/21/14
to twist...@googlegroups.com
On Tue, Sep 16, 2014 at 6:13 AM, Julian Steinwachs <julian.s...@fau.de> wrote:
My 2 cents: Torrents are not metadata secure. Distribution of the torrent creates direct connections between the author and the readers of that post. I'm not saying that torrents are a bad idea, but they dial down privacy significantly. They should definitely be optional.


Julian, good point.

But remember that you may use twister on top of TOR. So, once you get your new post/file distributed to at least one of your readers through TOR, you may completely disconnect from the network.

Further distribution of the content then doesn't require any direct connection to you.

regards,

Miguel

Сёма Мрачный

unread,
Sep 21, 2014, 9:55:23 AM9/21/14
to twist...@googlegroups.com

My proposal is a simple concept: if you follow some user, then you join his "media" torrent and helps sharing the files. If you don't follow, you don't host/share any of his data.
what if that user posts some stuff which you do not like and don't want to store?

I want to be able to mark (and unmark) each item as acceptable to store it. and it is not to necessary for me to follow someone to store certain parts of his media stuff. and also I want some page in client with list of all stored items to control them.

Miguel Freitas

unread,
Sep 21, 2014, 11:33:34 AM9/21/14
to twist...@googlegroups.com
No problem at all! That's quite some UI to design and implement, but i don't dislike the idea. I think whoever codes should be able to sort these details.

Of course, my suggestion of "follower joins the torrent" is the easiest to start with (almost no extra UI needed). File size limits and other limits all require extra work.

So please feel free to contribute a patch of this per-item storage idea when we have the basic support working ;-)

regards,

Miguel


Сёма Мрачный

unread,
Sep 21, 2014, 2:02:27 PM9/21/14
to twist...@googlegroups.com
So please feel free to contribute a patch of this per-item storage idea when we have the basic support working ;-)
sorry, I don't feel free yet. one day when I learn to code…

Miguel Freitas

unread,
Jul 30, 2015, 1:59:47 PM7/30/15
to twist...@googlegroups.com
Hi Denis,

Are you still around?

I have been thinking: reverting our heavily-patched libtorrent to work
with both attached files and twister posts is probably going to take
quite a bit of work...

What if, instead of doing that, we link again to another copy of
libtorrent (a system library provided by distribution, unpatched
upstream) just for handling the files users may want to attach?

twister executable code size in memory would increase, but not a big
deal. we would also need to open another port.

However enforcing per-user, per-file limits would be easier. Also,
completely disabling file attachment support doesn't disrupt what
already works (for example, torrent is not permitted in my
university's network)

Food for thought.

regards,

Miguel

Denis Ryabov

unread,
Jul 30, 2015, 4:32:12 PM7/30/15
to twist...@googlegroups.com
Hi Miguel,

Most likely both ways are necessary, because of silver bullet is out there.

Disadvantage of libtorrent's attached files is long rotation time after there will be thousands of files attached in the network.

And disadvantage of additional torrent is limited size of attachment (maximal piece size is limited to 128Mb, but I'd limit such an attachment to 8 or 16Mb for images and short audios only).

-------
Best regards,
Denis Ryabov

30.07.2015, 20:59, "Miguel Freitas" <mfre...@gmail.com>:

Erkan Yilmaz

unread,
Jul 31, 2015, 6:34:07 AM7/31/15
to twist...@googlegroups.com
>twister executable code size in memory would increase, but not a big deal.
>we would also need to open another port.

I'm currently on a weak device here:
so, I'm not sure what increasing twister's resources means exactly?
By how much % would e.g. the RAM resources increase by this?
I am assuming HD space will also be critical over time.

Also, my current environment doesn't enable me to open up ports :-(

All in all, it'd be great to have an option like:
"I don't want my device to do any work for feature X" (e.g. X = attaching files)

Erkan

--
Find me at: 

**********************************************************************************************
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Diese eMail enthaelt vertrauliche und/oder rechtlich geschuetzte Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese eMail irrtuemlich erhalten
haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht
gestattet.
**********************************************************************************************

Miguel Freitas

unread,
Jul 31, 2015, 7:32:18 AM7/31/15
to twist...@googlegroups.com
On Fri, Jul 31, 2015 at 7:34 AM, Erkan Yilmaz <erk...@gmail.com> wrote:
>>twister executable code size in memory would increase, but not a big deal.
>>we would also need to open another port.
>
> I'm currently on a weak device here:
> so, I'm not sure what increasing twister's resources means exactly?
> By how much % would e.g. the RAM resources increase by this?
> I am assuming HD space will also be critical over time.

I have no idea, i don't have these answers.

If you run a torrent client, how much RAM does it use?

Excuse me but imho this is quite nearsighted. we shouldn't refrain
from trying new technologies and ideas because they might not work
well on limited devices. Besides, libtorrent's resources usage may be
tweaked in a lot of different ways.

>
> Also, my current environment doesn't enable me to open up ports :-(

I can't use torrent in my work either, that doesn't prevent me from
seeing torrent as an useful tool for a lot of people...

> All in all, it'd be great to have an option like:
> "I don't want my device to do any work for feature X" (e.g. X = attaching
> files)

Of course!

External libtorrent wouldn't be a requirement for compilation.
Disabling at runtime should be fine as well (an unused mmapped code
shouldn't use RAM i think).

regards,

Miguel

Erkan Yilmaz

unread,
Jul 31, 2015, 2:50:57 PM7/31/15
to twist...@googlegroups.com
Hi Miguel,


>> All in all, it'd be great to have an option like:
>> "I don't want my device to do any work for feature X" (e.g. X = attaching
>> files)

>Of course!

>External libtorrent wouldn't be a requirement for compilation.
>Disabling at runtime should be fine as well (an unused mmapped code shouldn't use RAM i think).

It's OK for me, if I can turn such features off/on.
I just don't want my resources more used than what I have currently

Erkan


--
You received this message because you are subscribed to the Google Groups "twister-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to twister-dev...@googlegroups.com.
To post to this group, send email to twist...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Julian Steinwachs

unread,
Aug 1, 2015, 3:28:03 AM8/1/15
to twist...@googlegroups.com
Hi all,

I also would probably turn that feature off immediately. File sharing comes with a lot of legal issues. I don't want to get sued because someone i follow shares some copyrighted material that my client then redistributes automatically. With 140 characters this is much less of an issue.

If the main client should have such a feature nonetheless i would suggest to make twisterd interoperable with ipfs (http://ipfs.io/) and add a simple url field to every twister post.

Greetings
Julian

Miguel Freitas

unread,
Aug 1, 2015, 7:19:03 PM8/1/15
to twist...@googlegroups.com
On Sat, Aug 1, 2015 at 4:27 AM, Julian Steinwachs
<julian.s...@fau.de> wrote:
> Hi all,
>
> I also would probably turn that feature off immediately. File sharing comes
> with a lot of legal issues. I don't want to get sued because someone i
> follow shares some copyrighted material that my client then redistributes
> automatically. With 140 characters this is much less of an issue.

Good point.

> If the main client should have such a feature nonetheless i would suggest to
> make twisterd interoperable with ipfs (http://ipfs.io/) and add a simple url
> field to every twister post.

Exactly. I was thinking about adding the url field, which in case of
torrent would be a magnet link. ipfs url is a nice possibility as
well.

How is ipfs going? Are you using it?

regards,

Miguel

Julian Steinwachs

unread,
Aug 3, 2015, 2:51:20 PM8/3/15
to twist...@googlegroups.com
I haven't tried it out. I don't think that anybody is using it for
something else then experimenting. It would probably be best to ask them
directly whether they think its ready for adoption in a project like
twister.

I looked at ipfs a little bit more and found on the good side that they
have an API endpoint and good documentation for javascript side
interoperability. This means that twister-core wouldn't need to know
anything about ipfs. twister-html could upload the files to the endpoint
and embed the url (e.g. ipfs://S0m3W3eiRdH4sh...) into the post. On the
bad side: windows support seams to be non-existing.

Greetings
Julian
Message has been deleted

Denis Ryabov

unread,
Aug 9, 2015, 1:23:33 PM8/9/15
to twist...@googlegroups.com
Erqan,
 
There is an issue with such an implementation: assume one have Twister installed both on a desktop and a smartphone, and (s)he trying to post a photo from smartphone and then write a text post from desktop (because of it's more convenient), but it might take some time for rich message with photo to distribute over Twister network (mobile networks might be very slow and unstable), and text post might share the same "k" value(s). Currently, 3rdparty user loads one of signed posts in the case of such conflict, because of every "k" value corresponds to a single post, but in the case of rich messages it will be possible to load "k" from one rich message and "k+1" from another.
Message has been deleted

Julian Steinwachs

unread,
Aug 11, 2015, 2:57:08 PM8/11/15
to twist...@googlegroups.com
But thats not an issue of rich messages in particular, that happens with every message. When switching clients you have to wait until the messages propagate, or have other means to keep both clients in sync regarding your lastk.

I actually think that the rich message design is pretty solid. Regarding the implementation: Am i right that currently you can only download and redistribute all rich messages of you followees or none? If so i think it would be cool to have a download/redistributed-on-demand-mode at some point in the future. The UI would show the rich message without the content and only when i "open" it it would get downloaded. If i find the content inappropriate i could then also delete it and thereby stop the redistribution on my behalf. For me that would be the perfect way to deal with the legal/moral side-effects of file sharing.

But all or nothing is a good start. So i'm all for it.

Greetings
Julian (@tschaul)

On 09.08.2015 20:34, 'Erkan Yüksel' via twister-dev wrote:
that's right, that's a really issue..

Erkan Yüksel

9 Ağu 2015 tarihinde 20:23 saatinde, Denis Ryabov <dry...@yandex.ru> şunları yazdı:

--
You received this message because you are subscribed to the Google Groups "twister-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to twister-dev...@googlegroups.com.
To post to this group, send email to twist...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Message has been deleted

Frédéric JOUVIN

unread,
Jan 22, 2020, 7:54:25 AM1/22/20
to twister-dev
Hello Miguel.

I'm back on these discussions as I really need to have this functionnality implemented for my crypto-anarchist work with other crypto-anarchist friends arround the world.
We need a reliable way to communicate, and Twister is the only tool allowing that.

I think you latest approach written below is from far the best one. Keeping compatibility with what is existing is good, and the way you propose to do it is according to me the best too.

Do you have some news about this project advancement ?

Who is still ready to contribute to it ?

I'm okay to spend time on it. As I need it.

Kind regards,

Frédéric.
>>>  email to twist...@googlegroups.com.
>>>  To post to this group, send email to twist...@googlegroups.com.
>>>  For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "twister-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to twist...@googlegroups.com.

Miguel Freitas

unread,
Jan 22, 2020, 1:42:36 PM1/22/20
to twist...@googlegroups.com
Hi Frédéric,

I really have no resources (time) for implementing any of this, sorry.

regards,

Miguel

To unsubscribe from this group and stop receiving emails from it, send an email to twister-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/twister-dev/2a94699c-15e9-4d21-ae41-e240f13baddf%40googlegroups.com.

Frédéric JOUVIN

unread,
Jan 22, 2020, 2:54:08 PM1/22/20
to twister-dev
Thanks for this reply Miguel.

Indeed, would you at least have a few minutes to share with me to give me the entry points in your code, and a few explainations on how the hacked bitcoin lib works (What was changed), and same for the bittorrent lib, so that I can fully understand the mecanism you implemented with wallets. I still ignore how it works for account creation, and what kind of bitcoin transactions are generated and when.

I would also need to know what kind of objects are stored in the DHT, and with all this, I should be able to understand more easily your code and add what needs to be added to implement this new functionnality.

Entering your code with ZERO informations regarding the basic mecanism twisterd implements is very hard, but once ytou know what is done, it's way much easier.

I am looking forward to hearing from you.

I hope Denis will join my efforts in this regard. If we are a team of two, it's stimulating to move forward faster.

Kind regards,

Frederic, from Paris.
To unsubscribe from this group and stop receiving emails from it, send an email to twist...@googlegroups.com.

Miguel Freitas

unread,
Jan 22, 2020, 5:54:56 PM1/22/20
to twist...@googlegroups.com
Frédéric,

On Wed, Jan 22, 2020 at 4:54 PM Frédéric JOUVIN <frederi...@gmail.com> wrote:
Indeed, would you at least have a few minutes to share with me to give me the entry points in your code, and a few explainations on how the hacked bitcoin lib works (What was changed), and same for the bittorrent lib,

This is definitely *NOT* a few minutes explanation.

If you want to track what was changed from bitcoin you will need to follow the earliest patches starting at Jul 15, 2013, like when I added "username" and started removing "coins" from bitcoin. Here are they:

 
so that I can fully understand the mecanism you implemented with wallets. I still ignore how it works for account creation, and what kind of bitcoin transactions are generated and when.

I would also need to know what kind of objects are stored in the DHT, and with all this, I should be able to understand more easily your code and add what needs to be added to implement this new functionnality.

You can see DHT protocol and objects documented here:


 

Entering your code with ZERO informations regarding the basic mecanism twisterd implements is very hard, but once ytou know what is done, it's way much easier.

This is quite unfair to state you have ZERO information, I must say. There are lots of notes and explanations about how development progressed, there is a whitepaper with a complete overview, every new feature was documented on our "news" page. All these have been made available on twister.net.co from right from the start. All public information. 

You can't realistic expect that I'm going to explain you everything again "in a few minutes".
 
regards,

Miguel

Frédéric JOUVIN

unread,
Jan 22, 2020, 5:59:22 PM1/22/20
to twister-dev
Thank a lot for providing the links to these informations (I was scared they were not available).
I'm gonna read all this, and will be back to you if necessary.

Again, I want to thank you for all your work that has been from very far the only system that has resisted these last 5 years, and who is still on top of the pack, largerly ahead of bitmessage.

Kind regards,

Frederic.

Frédéric JOUVIN

unread,
Jan 23, 2020, 3:25:35 AM1/23/20
to twister-dev
Hello Miguel,

Sorry to disturb you again about documentation, but I couldn't find the White paper on the website (In the news section), could you provide me the link ?

Kind regards,

Frederic

Frédéric JOUVIN

unread,
Jan 23, 2020, 5:26:37 AM1/23/20
to twister-dev
It's okay, I found it.

And sorry about saying ZERO, you were right, all the needed information was published. My bad. It's just I has not found them.
Sorry for the involuntary offense.

Kind regards,

Frederic.

Frédéric JOUVIN

unread,
Jan 23, 2020, 6:10:08 AM1/23/20
to twister-dev
By the way, about P2P network security model in general,

I encourage all those interested in making Twister evolve, as I am, to read this post :

It's good example on how P2P overlay networks security model can be attacked thanks to massive distributed compromission of all nodes, that has dangerously rise these last 3 years because of the discovery and the publication of more or less 40 major architectural backdoors into microprocessors (SPECTRE, MELTDOWN, etc...) and a few other integrated circuits like DDRAMs.

The given example is focusing on Tor network itself, but the argumentation is indeed valid for all kinds of P2P networks, including Twister.

The best mitigation technic proposed consists of using a TPM (In this example, Intel SGX secure enclave) :


I really encourage everybody reading this paper.

Unfortunately, secure enclave have new flows that were advertized during the last CCC conference in end december 2019 :

The Intel SGX enclave has been broken thanks to the discovery of a new microprocessor architectural security breach perfectly described here :


This bad news about SGX broken shall not discourage the efforts of using SGX at all according to me, but shall stimulate Crypto-Anarchists to finding new ways to mitigate this new breach, either by software, or by hardware. FPGA processors are definitely, according to me, the best approach to mitigate this, thanks to RSIC-V processor implemented with FPGA, as an example.

Still, hardening Twisterd with SGX is a work that worth it.

I also think that another usefull functionnality to add to Twisterd would be the possibility to use hardware secure tokens like Yubikey, Nitrokey, or even LEdger Nano S to generate the key pairs and key private keys safe, and use such USB token to implement all crypto primitives, using the PKCS11 interface. This should not be a lot of work indeed, and it really worth it.

Another interesting thing to do would be to implement a GTK+ interface doing RPC on the twisterd daemon insteal of the HTML interface. This would enhance the overall security model too.

Having ready to use very simple to install and maintained packages for most Linux distributions, and also windows and Mac OS is still something really needed. Many contributors have worked on this already, and these efforts really need more support from the Twister community.

All those interested to forwarding Miguel's work on twister, please do contact me (@stman on twister) so that we can exchange and build an international team to work on those much needed improvements.

Kind regards to all, and my apologize again toward Miguel for saying there was no documentation available to continue his great work on twister.



And

Frédéric JOUVIN

unread,
Jan 23, 2020, 6:22:39 AM1/23/20
to twister-dev
For those who want to read more about the Plundervolt vulnerability, affecting Intel SGX can read the white paper about published here :


Please note that other securre enclaves, like ARM TrustZone, must be affected equaly according to me.

Kind regards to all and happy Crypto-Anarchist hacking.

Frederic.
Reply all
Reply to author
Forward
0 new messages