Hi
I briefly mentioned something about this on the bitcoin-dev IRC room. In general, it seems experts (like sipa i.e. Pieter) are against using sidechains as a way of scaling. As I only have a high level understanding of the Bitcoin protocol, I cannot be sure if what I want to do is actually defined as a side chain, but let me just propose it, and please let me know whether it can work, and if not why not (I'm not scared of digging into more technical resources in order to fully understand). I do have a good academic/practical background for Bitcoin, and I'm ready to contribute code if needed (one of my contributions includes a paper wallet creator written in C).
The main problem I see with increasing the block size limit is that it increases the amount of storage required to fully validate all transactions for say 100 years (a person's life). With 1 MB blocks, you can store all lifetime transactions on a 5 TB drive, which basically any regular user can do. With 10 MB blocks, you need a 50 TB drive, not accessible for regular users. Yes, it's possible that in the future hard drive technology will get cheaper and smaller, but this still didn't happen, so we can't just say "it should be doable at the rate of Moore's law etc...", we need to know that it is accessible for everyone, now. Also, don't forget that human life expectancy can increase with time as well. I know, it sounds silly to use a human lifetime as a measurement of how far back each user should be able to store transactions for, but what is a better measurement? This is a technology made for people i.e. humans, right, and the important part is that it is for regular people and not just well privileged people. You can search my last four emails for some more calculations.
What sipa told me on the IRC channel is that Bitcoin Core does not care about old transactions. It only looks at the current blocks. Yes, that makes sense, but how do you know that your machine wasn't compromised when validating the previous blocks? And what if you want to check some old transactions (assuming you didn't index everything)? What if some of your old transaction data was lost or corrupted? I think it is clear that it is useful to be able to validate all blocks (since 100 years) rather than just a pruned part. It empowers people to have as much information about Bitcoin transactions as do large data centers; transactions that may include government or corporate corruption. This is the key to how Bitcoin enables transparency for those who should be transparent (individual users with private addresses can still remain anonymous). Also, 5 TB takes about 20 days to sync starting fresh, on a regular computer, so it allows easy entry into the system.
So assuming we agree that people should be able to store ~ a lifetime of transactions, then we need 1 MB blocks. But of course, this leads to huge transaction costs, and small purchases will be out of limits. So to fix this, I propose adding a 10 1 MB chains below the main chain (sorry on the IRC room I said 10 10 MB chains by mistake), so effectively, you have a new 10 MB chain that is partitioned into 10 parts. You can also add a third level: 100 1 MB chains, and keep going like that. The idea is that when you make a large transaction, you put it through the top chain; when you make a medium sized transaction, you put it through one of the middle chains, which will be verified (mined) by the middle chain, and the top chain will verify the aggregate transactions of the middle chain. If you have a small sized transaction, you put it through one of the bottom chains, the bottom chain verifies it, the middle chain verifies the aggregate transactions of the bottom chain, and the top chain verifies the aggregate transactions of the middle chain. By aggregate transaction, I mean the net result of multiple transactions, and I suppose it can be 20 transactions belonging only to one "sibling" chain for level 2, or 200 transactions for level 3, etc...
Now, how does the system decide to which of the 10 chains the middle sized transaction goes to? I propose just taking some simple function of the input addresses mod 10, so that you can just keep randomly generating a wallet until you get one with only addresses that map to only one of the 10 chains (even distribution), so that someone can choose one of the 10 chains, and store only the transactions that belong to that chain. They should also choose a chain from level 3, etc... So in effect, they will be storing a chain with block size O(n) where n is the number of levels. They may store multiple sibling chains at one level, if they want to track of other people's transactions, such as those of their government MP, or perhaps, they want to have a separate identity that would be more anonymous with a separate series of sibling chains. This will increase the storage size, but the increase will be proportional to the number of things you want to keep track of (at least this kind of system gives you the ability to fine tune your storage needs to the level of "things" you want to keep track of). Also, note that there may likely be duplication of transactions, since transactions can include addresses that are associated with different silbling chains, but this effect shouldn't make a big difference, and again will depend on the complexity of the transactions you want to keep track of.
So how can this work? I propose that we keep the current chain as the top chain, and then create 10 level 2 chains that also store Bitcoin and the Bitcoin can be transferred between chains (I think this is the idea of sidechains). How can we incentivize people to keep mining on the level 1 chain? Perhaps force it into the (soft fork) protocol that anyone mining on level 2, has to also mine on level 1, and in general, anyone mining on level n+1 has to also mine on levels n,n-1,...,1. Also, level 1 will have the best decentralization, so there should be enough people paying fees to get transactions there for their large transactions that require a high level of security and trust. Even if people stop using level 1, any bitcoin you own in the level 1 chain, can be transferred to level 2, and still you have 1 MB blocks due to the partitioning scheme. How to prevent transactions from clustering on one or a few sibling chains in a particular level? Well the more empty chains should have lower fees, so that should incentivize people... Note: This system also allows for the fine tuning of the transaction size to security ratio. Yes the lower chains will be less secure, but a lower sized transaction does not need as much security.