This is a concerning aspect of Ethereum's strategy to push scaling to layer-2 networks: Ethereum is a heavily audited and tested protocol that runs an extremely decentralized network of diverse clients. L2s can be...an AWS instance running arbitrary buggy code. Much of the confidence in the "base layer" that people using Ethereum currently experience will be significantly undermined if mundane transactions wend in and out of different L2s.
Yes! I got burned by Optimism in another way. They tell you to point your applications at etherscan.io for transaction data/history, but then, on November 11 last year, the pushed an update that deletes all transaction history up to that point, which you need for taxes!
They swore they'd have the history restored on Etherscan by Nov 18th, but they still haven't. Only recently they pushed a workaround that lets you download the transactions as a CSV, but that lacks the critical data from your transfers of non-ETH tokens. And then, an alternative source does have that data, but only as a binary blob you have to run through a decoder and parse out yourself.
The crypto tax software, of course, doesn't know what to do with it.
(Even if your local client cached the transactions, most, like MetaMask, left out the critical data above.)
68 days till the filing deadline in the US!
I asked the maintainers how they planned to do their own taxes, and one of them claimed that he was separately recording all sales in a spreadsheet. I had to inform them that the taxable events include more than just sales, and, even under the most aggressive interpretation of tax law, you need the other data to figure cost basis.
Let this be a lesson: Maintain local self-hosted copies of any necessary data, don’t rely on third-parties maintaining it and keeping it available. Should be standard practice for anyone doing anything serious with cryptocurrencies but unfortunately users seem complacent enough that easily accessible tooling is still lacking in many places and you may have to DIY scripts for some parts.
Your situation is unfortunate but it sounds like no fault on Optimism or Etherscan here.
(BTW just to be clear: you’re talking about off-chain data that was never part of on-chain txes, and this binary blob comes from some Optimism operator? If it’s on-chain data its just a matter of doing the right queries)
Heyo, chiming in from the Optimism team here. You should be able to get non-ETH token transfers (ERC20 or ERC721) via the Etherscan CSV export feature. You may have to make a dummy ERC20/ERC721 transaction to get the page to show up. If that isn't working for you, feel free to hit me up on Discord (I think we already have a thread together) and I can help get you whatever data you need.
> The crypto tax software, of course, doesn't know what to do with it
I’m still trying to calculate proper basis and gains on transactions from 2018… tens of thousands of trades in various complex hedges across 7-8 different exchanges. I just aggregated everything in a single line item on my original return but of course I got audited T_T
If the history is completely gone, your government won't be able to find it either, so you can just fill in whatever you want to explain how balance A became balance B.
The discussion here used to be way more thoughtful, it's only been bad the last year or so.
I think the degradation of crypto discourse here was mostly a knee-jerk reaction to NFTs. "NFTs are stupid, so all crypto is stupid, because NFTs are crypto" - that was likely the thought process behind all the toxicity seen here.
After a year of playing around with crypto I think I'm appropriately both skeptical and excited, which is hopefully a good starting point for non-trolling conversations.
That's an issue with all cryptocurrency infrastructure though: projects need to be proven to demonstrate robust value and it's probably one of the most adversarial spaces in software. History has shown that hacks and exploits of projects hurt the price of the native taken but do not really damage the long-term earned trust.
Exactly this, it's a very adversarial environment with huge stakes for those that can exploit it. Even projects that have been around for months, years can get exploited which is why I'd recommend waiting a long time before putting non-trivial amounts into any smart contract or crypto related projects.
That's also a big plus for Bitcoin, because it's been around the longest and because it's so much simpler than more complex chains like eth, it's as secure as it gets.
Ethereum actually has almost no client diversity. The vast majority of nodes run the geth client (go).
Regarding the security aspects of L2s: they will of course not be anywhere near as robust as ethereum itself, but over time they’ll get better. However, they also don’t need to be as robust as ethereum given they effectively benchmark against the ethereum chain so while things could go wrong, the amount of damage will be very contained and as the ethereum mainchain scales the damage radius becomes ever more contained. Finally the bridges that are being implemented to move assets from ethereum to the L2s can implement emergency withdrawal mechanisms which allow users to get their assets out even if things go wrong.
Not perfect, but the tradeoff seems reasonable to me given the performance enhancement and the diversity of functionality that can be offered via many different environments.
Disclaimer: I’m quite possibly biased due to my company working on L2s.
Re: GP comment - From a "trust" perspective, there is a distinct difference to call out between the integrity of data on the platform, and the trustworthiness of the platform itself (i.e., the ability for centralized control of all data)
In an instance where an L2 is compromised, the potential impact is limited to the integrity of data that individual L2 was contributing to the overall platform.
Those transactions which demand absolute integrity will naturally tend to occur on L1, for this reason. Risk mitigation strategies will develop for those operating on L2 + bridged chains.
With zkrollups, you get an on-chain proof that the off-chain infrastructure did everything correctly. A contract can even verify that proof before updating the data on chain.
Same with Polygon their Ethereum L2+Sidechaining scaling solutions. Polygon is quickly building a reputation for solid secure code, mostly because their team kicks ass and is proactive.
For those interested in the specific risks of various L2s as they stand, L2Beat has the best overview: https://l2beat.com/?view=risk
While the various L2s are all pretty bleeding edge, the current state/alternative [1] is that a majority of the TVL is being bridged to alternate L1s, where the bridges are also extreme weaknesses [2]. There was the recent $320M Wormhole hack [3], the last record white-hat payout ($2M bounty on $850M at risk with the Polygon Bridge) [4][5], and $2.2B sits on Avalanche's Bridge [6] which is an EOA that is secured by literally 4 SGX machines. [7]
It's L2, but you can have different types of L2s. With lightning network, you're opening and closing channels with a counterparty using on-chain transactions, so each channel can be tied back to an on-chain transaction.
Before someone points out that it would require tons of on-chain transactions to onboard everyone onto it, you can batch thousands of channel open/closes into a single transaction with new protocol upgrades.
hm okay, room for nuance, there are at about a dozen L2 technologies in deployment right now, each with multiple competitors using a specific technology.
It is yes, and even though the lightning network is considered one of the more secure/safe L2 networks, even it has had bugs (now solved) that potentially could have caused everyone to lose all their money, if those bugs had been taken advantage of.
Can you provide source for this claim? I thought that infura was the dominant infrastructure provider for eth and if it gets taken down, a majority of the apps goes down too.
Correct me if I'm wrong, but with those L2 tricks the plusvalue of Ethereum gets kinda diluted... and there's already a heavy discussion on the "why should I use it at all".
Most L2s will require users to pay transaction fees in ETH. Some will have fee abstraction where people can pay with tokens, but the rollup themselves will still end up paying ETH on L1.
Ethereum will essentially be a settlement layer for rollups, and everyone will be doing their DeFi, NFTs, etc on the rollups which are almost treated like their own chains.
We're super greatful to saurik for writing up such a great analysis of what he found. If you want to hear some of our key takeaways as the maintainers of the network, you can check out our disclosure post here [1].
If you're wondering WTF Optimism is... we are building an optimistic rollup on top of ethereum. The basic idea is to de-couple blockchain computation from data availability and allow a new operator to exist called a sequencer which can accept transaction requests and submit the calldata to Ethereum Mainnet, but do the computation on Optimism Mainnet. There is an idea of a fault proof which means you can verify that the computation done on Optimism Mainnet followed the exact rules of the EVM, and you can prove this on Ethereum Mainnet. Our fault proof codebase, cannon, was built by another jailbreak legend (geohot) precisely with the goal of running Ethereum's battle-tested code and minimize the chances of bugs like this. It's some really cool stuff. If you're into compilers, VMs, and blockchains alike, check it out! [2]
The protocol is still in active development, it is not done yet, and that's exactly why we set up this bug bounty program. We think bug bounties matter, a lot, and we're proud to now become the record holders of the largest bug bounty payout in history, however we hope to very quickly be beaten by someone else. Developers like saurik, who we've gotten to know recently, are super important for this ecosystem to thrive. Building this stuff is hard, and we want the best hackers in the world to get rich breaking these protocols because if we succeed in this industry, this technology will be the backbone of the world's financial infrastructure — it needs to be secure. Everything we write is also MIT licensed and developed completely in the open.
Very happy to answer any questions, I'll check this thread for the rest of the day — AMA :)
I've been wondering what the hardware requirements for running Optimism's infrastructure are relative to just running a Mainnet node. If Optimism can process more transactions than the main chain, does that mean state growth is also much higher? How is Optimism thinking about this problem as it moves to decentralize the sequencer in the future?
This is a fantastic question for pretty much every scaling solution out there — as the initial engineering work on rollups finish, many of the fundamental scaling problems re-emerge on L2. Right now, our system's hardware requirements are very similar to L1 mainnet, but the state is growing.
There are two solutions in the future: statelessness, and block-producer/verifier asymmetry. Statelessness (and related concepts like state expiry) has been under active research in Ethereum for years, and we've recently started our own contributions with a new stateless Ethereum client [1]
The other part of the solution is to leverage asymmetries between the hardware requirements of block producers and verifiers. TLDR: this lets you have high HW requirements for sequencers, but still secure the network with laptops. Vitalik recently wrote about this; you can read that here [2]
I have a question: why did you make transaction data from before the Nov 11 upgrade unavailable? How hard would this have been? It's just serving the same immutable transactions that were there before, right? People were expecting these to be available for planning and tax reporting.
Even finding out about them after the fact was difficult because the cause of missing transactions wasn't made public on the user-facing site. For months the maintainers fielded questions from people on the discord that could have been satisfied by an announcement on the website. And even the announcements on discord came slowly.
The Nov 11 upgrade radically changed how Optimism's backend worked. Transactions after 11/11 are executed in a VM that's much closer to the EVM than before. It's still possible to run nodes that access these pre-11/11 transactions, but because of the way Etherscan and geth are designed, it's unfortunately not as simple as just serving the same data again.
Etherscan CSV exports are the best solution we had that didn't require significant modifications to Etherscan's backend. You should be able to use the CSV feature to export all of your relevant pre-11/11 transaction data (transactions and ERC20/ERC721 transfers).
While we did our best to communicate this months in advance on our twitter, blog, discord, and documentation, it's hard to reach everyone and we totally agree that this is not ideal. At the time, we had to prioritize progress, but we've since made a firm commitment to not to update the chain in this way going forward. So, this shouldn't be something people will need to worry about in the future.
He states that Optimism doesn’t have a native gas token and native currency, and eth balances are implemented using ERC20 tokens with OVM instead of the native balance mechanism
However the exploit is using selfdestruct to transfer and create the remaining balance to the target address, effectively creating new tokens out of thin air.
> This means that, when a contract self-destructs, its balance is BOTH given to the beneficiary AND ALSO KEPT. If the contract had 10 ETH, 10 ETH are CREATED from thin bits and handed to the beneficiary.
But I thought from this explanation that contracts don’t have a balance because ETH is stored in an ERC20 contract, and is set to 0. How can the contract have balance (10 ETH) to transfer on selfdestruct when optimism doesn’t have a native balance?
Would it help if that paragraph had said "10 OETH"'instead of "10 ETH"? (I am going to go change it regardless, as that is probably at least theoretically less confusing; but, like: is that sufficient?)
Whats the best way to replicate these states on localhost?
When using the L1s, it is easy to fork the current state of the network with Brownie and bang at smart contracts for free using fake gas on localhost. Reserving any advantage or unexpected behavior you find for the bug report, or redeploying it on mainnet for the bug bounty paying the gas just that one time
But with L2s in the mix, especially Optimism, how would one do the same? Would it be like two instances of Brownie in virtual environments? Kind of like having a cluster of microservices booted up in Vanguard on localhost?
Yeah, so to run your own Optimism full node--the "whole stack"--you need 1) a normal Ethereum full node of some kind, 2) an Optimism data-transport-layer service (which scrapes the L1 looking for L2 transactions and provides a web service to access just that data), and then 3) an Optimism l2geth instance (which is an Ethereum node modified to read its transaction batches from the DTL).
Speaking of "bug bounties", I use the term liberally as a euphemism for hacking these contracts and taking everything for yourself under the observation that company/community bug bounty systems are broken and undervalued for the value they provide. Although seen as a euphamism now, I think the term is accurate especially when looking at how bounty was used in the American frontier or Wild West.
You made $2,000,042 from this without any drama, in a quick timeline even though it was technically outside of the scope of the program! I think many in the hackernews audience would have liked to have known that from the get go. Many people ignoring blockchain would pivot immediately to at least doing smart contract bug bounty research on the side just from knowing that alone, learning the extremely lucrative and marketable skills in the process. If you formatted the article to the bug-bounty timeline to payout format. You should even show some people a material thing that what you bought with it, because many people still don't understand that this is analogous and convertible to money in your bank account especially at these convenient amounts.
How much could you have seized with this bug at the time?
Readit News
They swore they'd have the history restored on Etherscan by Nov 18th, but they still haven't. Only recently they pushed a workaround that lets you download the transactions as a CSV, but that lacks the critical data from your transfers of non-ETH tokens. And then, an alternative source does have that data, but only as a binary blob you have to run through a decoder and parse out yourself.
The crypto tax software, of course, doesn't know what to do with it.
(Even if your local client cached the transactions, most, like MetaMask, left out the critical data above.)
68 days till the filing deadline in the US!
I asked the maintainers how they planned to do their own taxes, and one of them claimed that he was separately recording all sales in a spreadsheet. I had to inform them that the taxable events include more than just sales, and, even under the most aggressive interpretation of tax law, you need the other data to figure cost basis.
Your situation is unfortunate but it sounds like no fault on Optimism or Etherscan here.
(BTW just to be clear: you’re talking about off-chain data that was never part of on-chain txes, and this binary blob comes from some Optimism operator? If it’s on-chain data its just a matter of doing the right queries)
See this random account for an example of exporting ERC20 transactions: https://optimistic.etherscan.io/address/0x9c1e0c67aa30c063f3...
I’m still trying to calculate proper basis and gains on transactions from 2018… tens of thousands of trades in various complex hedges across 7-8 different exchanges. I just aggregated everything in a single line item on my original return but of course I got audited T_T
I think the degradation of crypto discourse here was mostly a knee-jerk reaction to NFTs. "NFTs are stupid, so all crypto is stupid, because NFTs are crypto" - that was likely the thought process behind all the toxicity seen here.
After a year of playing around with crypto I think I'm appropriately both skeptical and excited, which is hopefully a good starting point for non-trolling conversations.
That's also a big plus for Bitcoin, because it's been around the longest and because it's so much simpler than more complex chains like eth, it's as secure as it gets.
Regarding the security aspects of L2s: they will of course not be anywhere near as robust as ethereum itself, but over time they’ll get better. However, they also don’t need to be as robust as ethereum given they effectively benchmark against the ethereum chain so while things could go wrong, the amount of damage will be very contained and as the ethereum mainchain scales the damage radius becomes ever more contained. Finally the bridges that are being implemented to move assets from ethereum to the L2s can implement emergency withdrawal mechanisms which allow users to get their assets out even if things go wrong.
Not perfect, but the tradeoff seems reasonable to me given the performance enhancement and the diversity of functionality that can be offered via many different environments.
Disclaimer: I’m quite possibly biased due to my company working on L2s.
I think that is slightly misleading, all the client diversity efforts is focused on Ethereum 2.0 now, as the old clients will be dead soon. [1]
This is the most updated stat I've found: https://twitter.com/sproulM_/status/1481109509544513539 (read the rest of the twitter thread too!)
Still not great though, but better at least.
[1] https://clientdiversity.org/
Re: GP comment - From a "trust" perspective, there is a distinct difference to call out between the integrity of data on the platform, and the trustworthiness of the platform itself (i.e., the ability for centralized control of all data)
In an instance where an L2 is compromised, the potential impact is limited to the integrity of data that individual L2 was contributing to the overall platform.
Those transactions which demand absolute integrity will naturally tend to occur on L1, for this reason. Risk mitigation strategies will develop for those operating on L2 + bridged chains.
While the various L2s are all pretty bleeding edge, the current state/alternative [1] is that a majority of the TVL is being bridged to alternate L1s, where the bridges are also extreme weaknesses [2]. There was the recent $320M Wormhole hack [3], the last record white-hat payout ($2M bounty on $850M at risk with the Polygon Bridge) [4][5], and $2.2B sits on Avalanche's Bridge [6] which is an EOA that is secured by literally 4 SGX machines. [7]
[1] https://defillama.com/chains
[2] https://old.reddit.com/r/ethereum/comments/rwojtk/ama_we_are...
[3] https://wormholecrypto.medium.com/wormhole-incident-report-0...
[4] https://medium.com/immunefi/polygon-double-spend-bug-fix-pos...
[5] https://gerhard-wagner.medium.com/double-spending-bug-in-pol...
[6] https://app.uniwhales.io/avalanche/bridge-tracker
[7] https://medium.com/avalancheavax/avalanche-bridge-secure-cro...
Before someone points out that it would require tons of on-chain transactions to onboard everyone onto it, you can batch thousands of channel open/closes into a single transaction with new protocol upgrades.
Can you provide source for this claim? I thought that infura was the dominant infrastructure provider for eth and if it gets taken down, a majority of the apps goes down too.
You can choose one of ~20 different free RPC endpoints: https://ethereumnodes.com/
This doesn't include private or paid RPCs or just running your own.
Ethereum will essentially be a settlement layer for rollups, and everyone will be doing their DeFi, NFTs, etc on the rollups which are almost treated like their own chains.
We're super greatful to saurik for writing up such a great analysis of what he found. If you want to hear some of our key takeaways as the maintainers of the network, you can check out our disclosure post here [1].
If you're wondering WTF Optimism is... we are building an optimistic rollup on top of ethereum. The basic idea is to de-couple blockchain computation from data availability and allow a new operator to exist called a sequencer which can accept transaction requests and submit the calldata to Ethereum Mainnet, but do the computation on Optimism Mainnet. There is an idea of a fault proof which means you can verify that the computation done on Optimism Mainnet followed the exact rules of the EVM, and you can prove this on Ethereum Mainnet. Our fault proof codebase, cannon, was built by another jailbreak legend (geohot) precisely with the goal of running Ethereum's battle-tested code and minimize the chances of bugs like this. It's some really cool stuff. If you're into compilers, VMs, and blockchains alike, check it out! [2]
The protocol is still in active development, it is not done yet, and that's exactly why we set up this bug bounty program. We think bug bounties matter, a lot, and we're proud to now become the record holders of the largest bug bounty payout in history, however we hope to very quickly be beaten by someone else. Developers like saurik, who we've gotten to know recently, are super important for this ecosystem to thrive. Building this stuff is hard, and we want the best hackers in the world to get rich breaking these protocols because if we succeed in this industry, this technology will be the backbone of the world's financial infrastructure — it needs to be secure. Everything we write is also MIT licensed and developed completely in the open.
Very happy to answer any questions, I'll check this thread for the rest of the day — AMA :)
Also, we are hiring! [3]
[1] https://optimismpbc.medium.com/disclosure-fixing-a-critical-... [2] https://github.com/ethereum-optimism/cannon/ [3] https://boards.greenhouse.io/optimism
I've been wondering what the hardware requirements for running Optimism's infrastructure are relative to just running a Mainnet node. If Optimism can process more transactions than the main chain, does that mean state growth is also much higher? How is Optimism thinking about this problem as it moves to decentralize the sequencer in the future?
There are two solutions in the future: statelessness, and block-producer/verifier asymmetry. Statelessness (and related concepts like state expiry) has been under active research in Ethereum for years, and we've recently started our own contributions with a new stateless Ethereum client [1]
The other part of the solution is to leverage asymmetries between the hardware requirements of block producers and verifiers. TLDR: this lets you have high HW requirements for sequencers, but still secure the network with laptops. Vitalik recently wrote about this; you can read that here [2]
[1] https://twitter.com/ben_chain/status/1488275978983915523?s=2... [2] https://vitalik.ca/general/2021/12/06/endgame.html
Even finding out about them after the fact was difficult because the cause of missing transactions wasn't made public on the user-facing site. For months the maintainers fielded questions from people on the discord that could have been satisfied by an announcement on the website. And even the announcements on discord came slowly.
Etherscan CSV exports are the best solution we had that didn't require significant modifications to Etherscan's backend. You should be able to use the CSV feature to export all of your relevant pre-11/11 transaction data (transactions and ERC20/ERC721 transfers).
While we did our best to communicate this months in advance on our twitter, blog, discord, and documentation, it's hard to reach everyone and we totally agree that this is not ideal. At the time, we had to prioritize progress, but we've since made a firm commitment to not to update the chain in this way going forward. So, this shouldn't be something people will need to worry about in the future.
https://twitter.com/saurik/status/1491821215924690950
Deleted Comment
As far as I could gather from a quick googling, this is the largest single bug bounty payout in history.
[1] https://portswigger.net/daily-swig/polygon-pays-out-record-2...
Deleted Comment
He states that Optimism doesn’t have a native gas token and native currency, and eth balances are implemented using ERC20 tokens with OVM instead of the native balance mechanism
However the exploit is using selfdestruct to transfer and create the remaining balance to the target address, effectively creating new tokens out of thin air.
> This means that, when a contract self-destructs, its balance is BOTH given to the beneficiary AND ALSO KEPT. If the contract had 10 ETH, 10 ETH are CREATED from thin bits and handed to the beneficiary.
But I thought from this explanation that contracts don’t have a balance because ETH is stored in an ERC20 contract, and is set to 0. How can the contract have balance (10 ETH) to transfer on selfdestruct when optimism doesn’t have a native balance?
I still have trouble understanding how this exploit worked
When using the L1s, it is easy to fork the current state of the network with Brownie and bang at smart contracts for free using fake gas on localhost. Reserving any advantage or unexpected behavior you find for the bug report, or redeploying it on mainnet for the bug bounty paying the gas just that one time
But with L2s in the mix, especially Optimism, how would one do the same? Would it be like two instances of Brownie in virtual environments? Kind of like having a cluster of microservices booted up in Vanguard on localhost?
Speaking of "bug bounties", I use the term liberally as a euphemism for hacking these contracts and taking everything for yourself under the observation that company/community bug bounty systems are broken and undervalued for the value they provide. Although seen as a euphamism now, I think the term is accurate especially when looking at how bounty was used in the American frontier or Wild West.
You made $2,000,042 from this without any drama, in a quick timeline even though it was technically outside of the scope of the program! I think many in the hackernews audience would have liked to have known that from the get go. Many people ignoring blockchain would pivot immediately to at least doing smart contract bug bounty research on the side just from knowing that alone, learning the extremely lucrative and marketable skills in the process. If you formatted the article to the bug-bounty timeline to payout format. You should even show some people a material thing that what you bought with it, because many people still don't understand that this is analogous and convertible to money in your bank account especially at these convenient amounts.
How much could you have seized with this bug at the time?