Forking twister to create a new web app.

[Brady Rose]

I've been looking though the twister/bitcoin c++ code for the past few days considering forking it to creating a derivative web application. 

I've programmed a number of applications in c++ in the past but never anything as complex as bitcoin in the past, so I am trying to get a better feel for exactly what an undertaking it would take to modify twisters source code, fortunately the application I have in mind won't have to be very complicated and should only require limited changes to achieve the primary objective.  I know, im being kinda vague on what exactly I want to do, but for not I've decided to not reveal my full plan until I've gotten a bit further.  For not I will just outline my main objectives in context to twister.

Creating a new genesis block to create a new blockchain:

Its my understanding that in order to fork bitcoin, litecoin, or twister one must create a genesis block by changing some settings in the code, recompiling it, then taking some of the resulting output, putting it back into the source and then recompiling that. Since twister includes litecoins script mining I can only assume that forking twister is more similar to forking litecoin than it is to forking bitcoin.  I'm looking for commentary on if creating a new blockchain for a forked twister is roughly the same as forking litecoin or if there are some particularities that are unique to twister that I need to familiarize myself with.   I've been looking at this thread here as a reference of things to change in the code :  http://bitcoin.stackexchange.com/questions/21303/creating-genesis-block

In addition to that, before I get carried away actually creating a new genesis block, I need to figure out if the fields I wish to store in the blockchain need to be completely established BEFORE the new blockchain is created.  If this is the case it would seem shouldn't worry about creating the new blockchain until I've worried about...

Creating additional storage fields, and modifying the sizes of existing fields:

OK, so my first objective is to change the size of the main message stored to something bigger, as 140 characters just isn't big enough for what I am thinking of, so my first concern is to modify the code to make it possible so that instead of a message being 140 characters, to be something more on the order of a page or two of text. In addition I would like to create another field that can contain 2,000 characters. All in all for this project I can't imagine any more than 5 kb data total among the possible fields needed.  I am though, beyond the scope of this particular product interested in the practical limitations on how much data a transaction can hold.

I have identified certain areas of twister/bitcoin that I believe might need to be changed to accommodate this.

1.  Chain parameters

 https://github.com/miguelfreitas/twister-core/blob/master/src/chainparams.cpp

CTransaction txNew;

        txNew.message = CScript() << string(pszTimestamp);

        txNew.userName = CScript() << string("nobody");
       // probably add a line like :
       taxNew.Additional = CScript() << string("nothing for now");

https://github.com/miguelfreitas/twister-core/blob/master/src/main.h

/** The maximum allowed size for a serialized block, in bytes (network rule) */
static const unsigned int MAX_BLOCK_SIZE = 50000;
/** The maximum size for mined blocks */
static const unsigned int MAX_BLOCK_SIZE_GEN = MAX_BLOCK_SIZE/2;
/** The maximum size for transactions we're willing to relay/mine */
//static const unsigned int MAX_STANDARD_TX_SIZE = MAX_BLOCK_SIZE_GEN/5;
static const unsigned int MAX_STANDARD_TX_SIZE = (140+16+512+32); //msg+user+key+spare

/** The maximum size of a blk?????.dat file (since 0.8) */
static const unsigned int MAX_BLOCKFILE_SIZE = 0x8000000; // 128 MiB
/** The pre-allocation chunk size for blk?????.dat files (since 0.8) */
static const unsigned int BLOCKFILE_CHUNK_SIZE = 0x1000000; // 16 MiB

/** The maximum size for spam messages */
static const int MAX_SPAM_MSG_SIZE = 140;
/** The maximum size for username */
static const unsigned int MAX_USERNAME_SIZE = 16;

2.  probably modify stuff to do with JSON so that the call that creates the transaction so it can pass the additional variables


3. Modify the actual functions that create the transaction



I plan on making additional edits to this thread to document this process. I'm sure it will be useful info for myself and others in the future. All commentary or suggestions on how I can meet the objectives outlined would be appreciated.


Brady.

[Brady Rose]

OK , here is what I have decided to go with in terms of changes to main.h
The rest I believe will stay the same. This should accommodate 7.81 kb per transaction, although my application as I currently see it probably won't use more than 5kb per transaction.  I suppose a little extra space will help out if I suddenly think of another field I just have to add.  I've doubled the username length aswell, and increased the spam message size.  If anyone thinks any of these choices are unwise, please let me know why.

https://github.com/miguelfreitas/twister-core/blob/master/src/main.h

static const unsigned int MAX_BLOCK_SIZE = 80000;
static const unsigned int MAX_BLOCK_SIZE_GEN = MAX_BLOCK_SIZE/2;
static const unsigned int MAX_STANDARD_TX_SIZE = 8000; //5000+2000+32+512+32 / msg+url+user+key+spare
static const int MAX_SPAM_MSG_SIZE = 5000;
static const unsigned int MAX_USERNAME_SIZE = 32;

[Brady Rose]

Made modifications to newpostmsg() and createSignedUserpost();  to accomidate an additional field named addy to contain a url that the userpost is referencing. 


https://github.com/miguelfreitas/twister-core/blob/4509ff7e3fec6bc0b5811ca3e30c0192f7338817/src/twister.cpp


Value newpostmsg(const Array& params, bool fHelp)
{
    // increase the prams numbers to accomidate
    if (fHelp || (params.size() != 4 && params.size() != 6))
        throw runtime_error(
            // added <address>
            "newpostmsg <username> <k> <msg> <addy> [reply_n] [reply_k]\n"
            "Post a new message to swarm");

    EnsureWalletIsUnlocked();

    string strUsername = params[0].get_str();
    int k = params[1].get_int();
    string strK = boost::lexical_cast<std::string>(k);
    string strMsg = params[2].get_str();
    // added param field for address field
    string strAddy = params[3].get_str();

    string strReplyN, strReplyK;
    int replyK = 0;
    if( params.size() == 6 ) {
        strReplyN = params[4].get_str();
        replyK = params[5].get_int();
        strReplyK = boost::lexical_cast<std::string>(replyK);
    }

    entry v;
    // [MF] Warning: findLastPublicPostLocalUser requires that we follow ourselves
    int lastk = findLastPublicPostLocalUser(strUsername);
    if( lastk >= 0 )
        v["userpost"]["lastk"] = lastk;

    // added strAddy, an additional field for storing a web address
    if( !createSignedUserpost(v, strUsername, k, strMsg, strAddy,
                         NULL, NULL, NULL,
                         strReplyN, replyK) )
        throw JSONRPCError(RPC_INTERNAL_ERROR,"error signing post with private key of user");

    vector<char> buf;
    bencode(std::back_inserter(buf), v);

    std::string errmsg;
    if( !acceptSignedPost(buf.data(),buf.size(),strUsername,k,errmsg,NULL) )
        throw JSONRPCError(RPC_INVALID_PARAMS,errmsg);

    torrent_handle h = startTorrentUser(strUsername, true);
    if( h.is_valid() ) {
        // if member of torrent post it directly
        h.add_piece(k,buf.data(),buf.size());
    } else {
        // TODO: swarm resource forwarding not implemented
        ses->dht_putData(strUsername, "swarm", false,
                         v, strUsername, GetAdjustedTime(), 1);
    }

    // post to dht as well
    ses->dht_putData(strUsername, string("post")+strK, false,
                     v, strUsername, GetAdjustedTime(), 1);
    ses->dht_putData(strUsername, string("status"), false,
                     v, strUsername, GetAdjustedTime(), k);

    // is this a reply? notify
    if( strReplyN.length() ) {
        ses->dht_putData(strReplyN, string("replies")+strReplyK, true,
                         v, strUsername, GetAdjustedTime(), 0);
    }

    // split and look for mentions and hashtags
    vector<string> tokens;
    boost::algorithm::split(tokens,strMsg,boost::algorithm::is_any_of(" \n\t.,:/?!"),
                            boost::algorithm::token_compress_on);
    BOOST_FOREACH(string const& token, tokens) {
        if( token.length() >= 2 ) {
            string word = token.substr(1);
            boost::algorithm::to_lower(word);
            if( token.at(0) == '#') {
                ses->dht_putData(word, "hashtag", true,
                                 v, strUsername, GetAdjustedTime(), 0);
            } else if( token.at(0) == '@') {
                ses->dht_putData(word, "mention", true,
                                 v, strUsername, GetAdjustedTime(), 0);
            }
        }
    }

    hexcapePost(v);
    return entryToJson(v);
}





/*
"userpost" :
{
"n" : username,
"k" : seq number,
"t" : "post" / "dm" / "rt"
"msg" : message (post/rt)
// comment-chain mod, add addy
"addy" : address we are commenting on,
"time" : unix utc
"height" : best height at user
"dm" : encrypted message (dm) -opt
"rt" : original userpost - opt
"sig_rt" : sig of rt - opt
"reply" : - opt
{
"n" : reference username
"k" : reference k
}
}
"sig_userpost" : signature by userpost.n
*/

bool createSignedUserpost(entry &v, std::string const &username, int k,
                          std::string const &msg, // either msg.size() or
                          // comment chain mod, adding addy
                          std::string const &addy,
                          entry const *rt, entry const *sig_rt, // rt != NULL or
                          entry const *dm, // dm != NULL.
                          std::string const &reply_n, int reply_k
                          )
{
    entry &userpost = v["userpost"];

   
    userpost["n"] = username;
    userpost["k"] = k;
    // comment chain mod, adding addy
    userpost["addy"] = addy;
    userpost["time"] = GetAdjustedTime();
    userpost["height"] = getBestHeight() - 1; // be conservative



    if( msg.size() ) {
        //userpost["t"] = "post";
        userpost["msg"] = msg;
    } else if ( rt != NULL && sig_rt != NULL ) {
        //userpost["t"] = "rt";
        userpost["rt"] = *rt;
        userpost["sig_rt"] = *sig_rt;
    } else if ( dm != NULL ) {
        //userpost["t"] = "dm";
        userpost["dm"] = *dm;
    } else {
        printf("createSignedUserpost: unknown type\n");
        return false;
    }

    if( reply_n.size() ) {
        entry &reply = userpost["reply"];
        reply["n"]=reply_n;
        reply["k"]=reply_k;
    }
    //

    std::vector<char> buf;
    bencode(std::back_inserter(buf), userpost);
    std::string sig = createSignature(std::string(buf.data(),buf.size()), username);
    if( sig.size() ) {
        v["sig_userpost"] = sig;
        return true;
    } else {
        return false;
    }
}

[Brady Rose]

Creating the genesis block has been elusive to me up until now. I've managed to find the code that was used in other iterations of litecoin derivatives that others have used to generate a new genesis block.  Ive recently forked twister on github and have committed my changes to it. 

The code for comment-chain is here:

https://github.com/bradyrose/comment-chaind

The code for the firefox addon is here:

https://github.com/bradyrose/comment-chain-firefoxAddon


So so far I've only taken baby steps.  I'll be working on getting two copies networked an mining.  Once I've mined a few blocks I'll take a shot at using JSON-RPC to trigger postnewmsg, to store a longer than normal post along with the address field.  At that point I will need to start rewrite some of whatever functions might be involved in displaying what I've stored. 

Again, it would be great to get any sort of feedback!   Its lonely in this thread!


Brady.

[Neil Haran]

Hi Brady,

Were you successful in your attempt to fork twister? What is your email? We can chat further.

[Joe N]

Hi Brady,

I'm wondering how your attempts went in forking the chain and building your program. Please give me an update if you see this because I am trying to learn as much as I can about blockchain technology so that I can build a Dapp on top of Ethereum. Thanks

-Joe