BSV Devcon China: Miner fee calculation and merchant API – new tools for the Bitcoin SV network

Just another WordPress site

BSV Devcon China: Miner fee calculation and merchant API – new tools for the Bitcoin SV network

Hi, everyone! I’m Lin Zheming, the founder and CEO of Mempool and DotWallet You can call me Miner Zheliang I graduated from the School of Economics at Renmin University of China, and began to learn about blockchain in 2011 Then, I founded Fuzhou Boquan Network Technology Co., Ltd. in 2016 It’s a technology innovator dedicated to the global public blockchain based on Proof of Work, and committed to the verification and processing of blockchain transactions, as well as to the research and development of smart contracts At the beginning of 2020, the company became one of the first Bitcoin, BSV and BTC mining pools in China to be filed with the Cyberspace Administration of China for blockchain, and obtained a Value-added Telecommunications Services license issued by the People’s Republic of China With an innovative token-less blockchain usage paradigm, and combining the credit advantages of global public blockchain, we provide a simple, easy-to-use, safe and stable Bitcoin blockchain underlying infrastructure for global developers and enterprises that are exploring legal uses of blockchain The founding team consists of early blockchain builders in China possessing rich experience in professional products and R&D And as the leader, I am an ambassador for the Bitcoin Association in China and a founding member of the Bitcoin SV Technical Standards Committee We are devoted to promoting social progress with the only global bitcoin ledger whose source of truth cannot be tampered with Now I’m going to start the keynote speech The speech is about Merchant API, covering what a Bitcoin node is, the competition mechanism of miners, how Merchant API works, and its usage scenarios It is designed for developers and enterprises who want to develop with public blockchain, as well as those who are interested in Bitcoin SV miner competition A big welcome to all! When it comes to Bitcoin node, we need to look at it from two aspects: namely, the role positioning and the incentive system of the node In terms of the role positioning of Bitcoin node, we can see that mining is the process of verifying, packaging and keeping account of transactions It is a node only when there is mining and there must be mining in a node By miners we mean transaction processing service providers that operate nodes, rather than operators that mainly provide hash power In this case, the miner does several things First of all, it verifies all the transactions it receives, to confirm that each transaction is legal and valid; then, it packages these transactions into blocks, to form a transaction template to be processed The template will then be submitted to the operator providing hash power, which will calculate the transaction template, so as to find a suitable proof of work that can be verified During the process, the miner needs to charge a fee for each transaction processed, which is used to cover its system cost, and serves as reasonable profit Miners resolve the conflicts between each other and help reach a consistent state through proof of work These are the three main things a miner has to do So, in this process, why do miners do such things? What’s the essential motivational driving force? OK, let’s take a look at the incentives of these nodes There is an important driving force for every miner participating in the Bitcoin network: to package more transactions and get higher fees These fees are all provided to miners by those who generate the transactions In addition, these miners need to reduce their orphan rates, about which I won’t elaborate here To put it simply, if your block is more easily accepted by other miners, your orphan rate will be lower So it’s very important for each miner to win other miners over to vote on the transactions and blocks assembled by it,

thereby stimulating the mining by nodes During mining, it is clear the more transactions the miner packages, the more fees it can collect, and the higher profit it may get in this process As long as the single transaction fee exceeds the cost borne by the miner for processing the transaction, the miner can obtain a certain marginal profit, and this is profitable Until the time the fee the miner receives for packaging a single transaction is equal to the cost it has to pay for the transaction, the miner may consider not accepting or packaging the transaction OK, now there’s a problem Many transactions don’t know what fee rate, or transactions with what fee rate, the miner accepts to cover its cost, and why the costs of these transactions are variable This has something to do with the orphan rate I mentioned As the number of your transactions grows, when a miner communicates with other miners in its block, its block should be sent to all other miners for verification The larger your block capacity is, the higher the transmission and verification cost will be, and when each miner receives the blocks from other miners, it can only start a new round of mining after completing block verification So, in this case, if the number of transactions received by a miner is too large to exceed the average processing capacity of the network, it is very likely these blocks will be at a disadvantage in the block competition Before the transaction verification is completed, another miner may have packaged a competitive block of the same height, which can be verified more easily So, in this case, many other miner nodes will choose to vote on a block that can be verified more easily So the competition between miners is not all about packaging blocks first It’s also about making their own blocks the first to be recognized by all other miners This node is very important In terms of the orphan rate, the more transactions you receive, the higher your verification costs will be and other miners will need more time and costs to verify your block So there will probably be competing blocks during this period When talking about the cost of each transaction, it not only includes the processing cost, but also the loss that comes with the increase in the orphan rate caused by the cost of processing these transactions, which is the opportunity cost We can see that, on the one hand, in order to get more processing fees, the node has to hold more transactions and charge a fee for processing these transactions On the other hand, the block packaged cannot be too large because packaging a large amount of transactions will lead to a significant rise in the orphan rate of your block So in this case, given the opportunity cost, you will not package a particularly large block This is the current competition between nodes in a constrained environment How can we increase the transaction volume we package? And how can we reduce the orphan rates of our blocks? This brings in the competition mechanism for miners It can be divided into two types: one is to improve the network connectivity, and the other is through service differentiation Better network connectivity means other miners can receive the block you send faster and more easily In that way, they can complete the verification after receiving it

So these miners have an incentive to improve the connectivity between themselves and other miners But this doesn’t mean that, as a miner, you need to or your best option should be to keep the best connectivity with each miner who generates blocks Instead, a miner often needs to seek other miners with a larger share of hash power It needs to first maintain better connectivity with miners with a greater proportion of hash power; namely those who are more likely to be written into the next block and become bookkeepers of the blockchain A miner can, based on its observations of the network, and through proof of work, find miners of higher value, and then establish more direct and faster connections with them, so that its block can be passed to these miners more quickly And it can receive and verify the blocks sent by other nodes sooner After completing the verification, the miner can start mining, i.e., processing transactions In addition to better connectivity, faster verification is also necessary This is also a very important incentive for miners Why are we confident the Bitcoin SV network is capable of carrying all the transactions worldwide? It’s because, driven by the economic benefits, every miner will find ways to improve its transaction verification and processing capabilities So what does this mean? It means that, as an individual, by improving the verification capability, a miner can verify blocks mined by other miners faster, so as to start block processing sooner It also means that the processing capability of the entire network is increasingly improved with competition So if your processing capability is slower than the average of the network, you are likely to be kicked out of this network When your processing capability falls behind, you can’t keep up with other miners in terms of transaction processing speed, which means you will always be mining on lagged blocks, resulting in a higher orphan rate Orphan rate is a very important incentive mechanism on the Bitcoin network We shouldn’t think that the orphan rate is terrible It’s a natural thing, and it may happen to every miner, so it’s fair You can reduce the orphan rate only through investing more in this system, and making your block the first to be accepted and recognized by other block processing service providers; namely, other miners This serves as a competition incentive for miners with respect to network connectivity and transaction processing capabilities The orphan rate is there to create an incentive Apart from that, miners also try to improve their services in order to obtain higher returns For this purpose, they have to reduce their costs Some of you may know many operators who provide hash power carry out hashing operation, commonly known as mining, in places where the costs are very low; not only electricity costs, but also labor costs However, in addition to reducing the costs with respect to hash power, they also consider how to make good use of these data, in addition to mining, in order to get higher returns So where do these benefits come from? These are extra gains obtained by miners through providing paid data services It doesn’t mean only miners can provide these data services Rather, because they’ve already had this data, and they are going to process massive data across the network,

their costs of data storage and processing are sunk costs, which are necessary for mining Providing some additional value-added data services will give miners a great cost advantage These extra gains can increase the total income level of miners In this respect, miners are not merely service providers who just process transactions and mine blocks to get block subsidies and transaction fees They also need to create more value by operating the transactions and the data they are processing On the whole, as the total earnings of miners are of great importance in the competition, miners with a low yield will gradually be left behind in the network due to insufficient yield Because in processing the same transaction and data, some miners are profitable, while others are not The difference in their costs and earnings leads to the yield difference As a result, miners are differentiated In this case, every miner competing in the mining will strive to maximize its earnings And this promotes the competition and cooperation in the Bitcoin network And, lastly, there is a very important view that in the near future, non-miners may not be able to connect to the miner network to request block and transaction data from miners This is quite interesting and many people find it problematic You said that the Bitcoin network should be free to come and go from, and would require no permission, but why do you exclude new competitors? In my opinion, this is also based on the analysis of costs and earnings By providing data to other miners with hash power, a miner can speed up the verification of its future block by other miners, thereby reducing its orphan rate However, miners that do not mine, or nodes without accounting rights, are not able to reduce the orphan rate after receiving these data So, in this case, there are no strong incentives for miners to send their data to nodes that do not mine and have no accounting right for free In the future, those so-called full nodes will need to purchase data from miners to be eligible to join the miner network However, from another perspective, will this harm the competition incentives for miners? I personally don’t think so Why? Because new every miner that joins the network may carry out its first mining operation without all the transactions In other words, it may package empty blocks by mining only the block header This also makes sense Because, first of all, as the hash power of the new miner is just getting into the empty block, it will not cause damage to the block and transaction processing speed in the network Secondly, by sending out such a workload signal, the miner shows to other miners it is a participant capable of accounting, and provides its contact information for other miners to contact it This may be covered in the Merchant API I will talk about later today Given the new miner’s contact information, other miners are encouraged to establish connection

and exchange data with it, in the hope of reducing their orphan rates, as everyone then knows the new participant has enough hash power If my block can be recognized by it, I can reduce my orphan rate Seeing it from a selfish point of view, it’s very easy for everyone to embrace new competitors and to connect with them So how should applications cooperate with such a network of miners? We know that the competition between miners is very fierce, and that every miner is trying to reduce its costs and package more transactions These low-cost measures all come out of competition So developers and common applications should consider services generated by better competition mechanisms, as they are highly substitutable and competitive Therefore, each application needs to think about how to use these common services that can be directly provided by miners These services are often the basis of what we call simplified payment verification or SPV mode So when designing an application, we should bear in mind that: OK, I should design my business model and product as an SPV application But our infrastructure still has many weaknesses now, and we haven’t been able to serve these developers very well At the same time, the current SPV development still lacks some good paradigms and infrastructures for everyone to use easily But you may, based on the basic paradigm of these SPVs, think about reducing your infrastructure costs by using the services provided by service providers A simple example would be, as an SPV, you have to independently verify every transaction you receive But by what means? By the Merkle proof of the Merkle Tree, as well as the block header data And how can an SPV wallet get the latest block header data? Does it need to write a P2P client by itself, connect it to the miner network, and request the latest block header data from the miner network? In this case, do application developers need to be familiar with and understand Bitcoin’s P2P protocol in order to do so? It is not necessarily true First of all, we should know it is possible to understand and use the P2P protocol to obtain the block header data We believe that works, OK! But you don’t have to solely rely on this model You may obtain block header data from other application service providers, and independently verify the difficulty of this block header and the subsequent proof of work In this way, once you find the service providers you use have stopped working, you can switch to other providers at a very low cost This is because these service providers all provide underlying services, and they all have high levels of competitiveness In addition to such simple things as obtaining block headers, there is also Merchant API, or MAPI, M-A-P-I, to help applications communicate with nodes We’ve just said that the cost, competition and network connectivity may vary from node to node And with respect to the selection of rate and way of obtaining higher earnings, they may also have different strategies So, for applications, if you want your transactions to reach, for example,

more than 95% of the nodes with accounting capabilities across the network, it’s very difficult for you to structure your transactions by guessing what rate each node or miner will accept In light of this, a very important function of Merchant API will prove helpful, namely fee rate query You can directly ask the operator of each node what rate they accept, and then structure your own transactions based on the response you get These node operators may be listed in order of the accumulated hash power, or sorted by hash power from high to low, from which you may select the most suitable rate range, as per the hash power recognized range you need, in order to structure your own transactions As we all know, this is the easy part, and there are third-party service providers that can help application developers communicate with these operators It is important to note the use of the Bitcoin network at the application level is progressive If you are not satisfied with the service quality and attitude of these service providers, you can just drop them and try to do it yourself In that way, you don’t have to worry about your data and work or business model being strongly bound to them, which would otherwise cause your business to be controlled by others in the end OK, I just mentioned the fee rate query function of MAPI, and now I’m going to talk about how MAPI works, and how it integrates with applications What MAPI currently does, for one thing, is to replace the rate check module in the node, as well as to sign the returned results by using the miner’s identity As everyone knows, to verify transactions in a node requires many steps, including verification of the transaction script, the legality of the input used by UTXO in the transaction, and, last but not least, ensuring that the rate paid by the transaction meets the requirements Previously, all of these were done in the node with internal codes However, MAPI has taken a very important step It has stripped this rate check module from the codes of the full node, enabling different rates to be checked with independent codes In addition, it offers a new feature that combines Miner ID, a credit system endorsed with proof of work, so the miner’s identity may be used to sign the returned results Why do we need to sign the returned results? In this way, you can confirm that the results come from the miner to whom you send the data, and you will know who sends you the results But it plays some more important roles, and we can talk about this more later In terms of meaning, it is important to know that MAPI actually provides miners with a tool of price discrimination But price discrimination is not a derogatory term here In fact, it’s a neutral measure, through which we can provide services and billing methods of various levels and rates for different types of customers and usages For example, we can distinguish trusted customers from untrusted ones For trusted customers, we can set a higher blockchain-based transaction processing cap In this case, some application developers,

such as the well-known Weather SV, may write their data on the blockchain ledger with many blockchain-based transactions to obtain the proof of timestamp in the blockchain However, as most of its transactions are chain-transactions the cap for such transactions in the node is generally set to 25, and if the unconfirmed chain-transactions exceed 25, it will probably not be accepted by the block In this case, Mempool will provide some miners with larger blockchain-based transaction processing capabilities, which will not be available to all users, because it may lead to attacks on nodes and operators You may easily structure a bunch of transactions that are actually invalid, as the capacity of the chain-transactions is so large that your attack will be very strong and effective, and the attack costs will be low In this way, the miner may need to constantly verify transactions, and then store them in the mem-pool or move them out of it, as they are no longer valid, and will affect the receiving and processing of normal transactions by the miner So the miner is likely to grant different levels of permissions to different customers For example, the miner will grant higher authority to trusted customers, because it believes these customers will not attack it, while for untrusted ones, as the miner doesn’t know where these transactions are from, it might look at them in a cautious manner At the same time, we can stimulate different usages with MAPI Miners’ costs, such as bandwidth, server, and hash power costs, are relatively fixed Therefore, the more transactions I receive now, the more processing fees I can get Then I may offer a certain discount to some applications that can provide a huge number of transactions, provided that they can really provide such a large number of transactions So this will be profitable for me in terms of average cost As a result, we may offer different rate settlement methods for different types of applications and usage standards, to meet the needs of those users who cannot afford higher rates, but can generate a great number of transactions and create value But for many common application developers, the current rate is low enough that they can easily complete their tasks So this is price discrimination based on usage We can also do this based on different roles, including data service providers who may want the full amount of massive data, common wallets that may only need to send transactions to obtain such basic services as Merkle proof, developers of complex applications who may need miners to carry out some preprocessing and screening operations, in addition to providing them with basic data, and more importantly, connections to other miners I may find other miners, and negotiate with them through MAPI All the data we send can be encrypted for different miners, so as to push specific transactions to them Why do we do this? It’s because if we still send data through traditional P2P network, the general rate and transaction verification will be required However, this check process may be bypassed

if we communicate with other miners through MAPI As a miner, I have established credibility with other miners So in this case, we are aware the transactions I send may be more reliable, and will not attack other miner nodes Therefore, different nodes will pass information to each other and synchronize the blocks through communication methods such as MAPI So what’s the real meaning of MAPI? The central one is that it is a Trust and Verify model Just now we said that MAPI, on the one hand, adds the miner’s own signature, and on the other hand, offers a way of communication between miners and other miners and partners So, we will first trust our partners and believe that they will not attack our services through the back door After sending many low-rate transactions, double spending will increase the mem-pool reconstruction costs after the block is generated They may also send a large number of invalid transactions without ancestor transactions to flood the miner’s mem-pool Both of them are attacks on services For trusted partners, we may remove the guard against attacks to reduce the processing costs In other words, if you are a trusted partner who has had dealings with me for a long time, I’ll grant you a direct pass without checking many of your operations, but afterwards, we will verify these operations This means that once you abuse this communication channel in this way and attack us, we can effectively detect and prevent it So, in this case, miners’ loss is very limited, as we are able to detect such attacks and verify the behavior of partners rapidly, and then prevent illegal attacks in a simple way by not doing business with you anymore because you lied to me This model actually encourages honest behavior To this end, miners need to verify the information they return by signing it, as we said earlier Because when signed by the miner with its own Miner ID, the returned information is more effective, and the victim may then use this transaction information as evidence of being attacked This is why we believe that the Miner ID system based on proof of work is an excellent credit establishment system It is not only because you are inspired to participate honestly, but also because once you have done something wrong and negatively affected other users, you will be held responsible Based on MAPI, in addition to introducing more flexible charging mode through price discrimination, we can also introduce more advanced transaction features, such as a higher chain-transaction cap, or a lower cap of transaction dust Both ways were used by miners to defend against attacks As we all know, Satoshi Nakamoto once said the rate is essentially introduced for flood control

and preventing such a great measure from being abused After a trust relationship has been established, we can gradually remove these precautions previously used to prevent attacks, so that the ledger can be fully used at a lower cost At the same time, miners may also negotiate with each other and collaborate to provide honest businesses with more valuable and robust blockchain underlying services at a lower cost In addition, MAPI has a very important feature: monitoring and protection based on double spending I’ve explained it in detail in a previous Webinar, so I won’t elaborate on it here One important feature of MAPI is that it can help all the developers and applications make full use of the infrastructure between miners to confirm and protect instant transactions under an established security model You don’t have to figure out the concept of block in blockchain accounting All you need to care about is whether your transactions reach all the bookkeepers, and that you haven’t received any attempts of double spending before that In this case, your transactions may be considered a success And this can be done only through MAPI, as you still need a statement from the miner showing who is responsible for this, i.e., the receipt with the miner’s signature, to ensure that your transactions are indeed accepted by the miner Otherwise, you might be deceived by some intermediaries Now I’m going to talk about some usage scenarios for MAPI First of all, it’s the miner’s interface, which has the following features: protection against double spending, Merkle proof, and real-time notification All of these can be achieved through the services directly provided by miners In terms of protection against double spending, it’s not just the notification of the current state that counts On the basis of that, we may also provide a notification of the current double spending transaction, unlike what the nodes do now: discarding the second transaction directly after receiving a double spending transaction The detection and notification of double spending transactions is a very important service However, not every node operator will detect and send notifications of double spending for each transaction They often send notifications for important transactions subscribed and looked after by those who generate and send these transactions And such notifications may be direct feedback to the querying party through a real-time notification channel Besides, we can provide Merkle proof to prove that a transaction is included in the block Every developer program that develops Bitcoin applications no longer needs to connect to a so-called full node in the background Because of simplified payment verification and Merkle proof, you can easily tell whether your current transaction is included in the block or not For further details on how to use SPV and these infrastructures to help applications, please stay tuned for the presentation of Zhang Wei from nChain at this DevCon

Here is an example from an existing miner of what the MAPI interface looks like There are several fields in it, including payload, which is the returned information to be sent You can load your needs through the payload, and results returned to you by the miner will be displayed in it It also includes the signature and publicKey, which may be used together to verify the content of the payload, so as to ensure that it has not been tampered with during the process In addition, it includes encoding, i.e., its encoding format, as well as mimetype, both of which are used to define how the payload is parsed When calculating the miner rate, we use a method called DTFC steps, which utilizes a given solution to provide each application developer with a way to find out what rate it should use now through the MAPI interface It has two important features Firstly, it applies the integer division operation, and secondly, it has a solution for zero rate It doesn’t mean the rate is unlikely to be zero, but if the rate is 0, it will be set to 1 When setting the rate, it is recommended to calculate it with the decimal system instead of 1024 These are the interfaces that miners can currently provide So how to use them? First of all, you can use the MAPI access provided by Mempool through the DotWallet for Developers platform And TAAL is another well-known Bitcoin SV miner, which also provides an API interface You can also access it through the API provided by MatterPool, a very small miner All of these three services are provided by miners with hash power Apart from these, there are encapsulated SCKs and APIs that can make it easier for existing developers to use the infrastructures provided by miners, including MinerCraft from _unwriter, the developer API provided by WhatsOnChain, as well as the latest Txbox from Libs All of them are very easy to use and are great tools for beginner developers On top of this, we may provide more and more rich services at the application level of MAPI in the future They are not just limited to those services related to rate and transactions I just mentioned, but also include the inter-node connection and transaction transmission, as well as payment processing service providers With respect to a collection agency, you can regard it as the acquiring bank on the POS machine of a bank It is there to help process payment transactions We may also provide such blockchain data services as blockchain data query and various applications They may communicate with miners via MAPI, and have important data transmitted by these miners In addition, we may offer certain credit guarantee services based on the proof of work or the Miner ID All of these are the types of services we can explore I also hope this will bring you some inspiration to find out what problems we can solve now with the MAPI mechanism That’s all for my presentation Thank you for watching If you have any questions or if you are interested in the project, please contact us in the following way Our business email is [email protected]

You are all welcome to recommend this DevCon to blockchain practitioners around you Please stay tuned for the Bitcoin SV Developer Zone on CSDN, whose domain name is Thank you for participating in the first Chinese-language Bitcoin SV DevCon Later on, there will be other guests who will give more wonderful presentations Please don’t miss them and keep watching Thank you!