Protocol Reviews are quality assessments that scrutinize major key elements of deployed DeFi products to produce an overarching safety rating that indicates the quality of development processes and accompanying documentation.
The premise behind these reviews is that developers following best practice in software development should have secure code and maintain security that users can trust, as potential blind spots have been accounted for.
DeFiSafety’s PQRs are conducted according to a clear, standardized and replicable process. This process is well-documented and publicly available, so that readers can dig into the details of the review to see how the score was generated.
Members of the DeFi Community
Written in clear and accessible languages, Protocol Reviews serve to build confidence in DeFi, raising standards of security and transparency, and providing value to all members of the DeFi community.
Users
Clear, concise and detailed ratings give users a more comprehensive understanding of the protocols they’re investing in. This allows users to make informed decisions regardless of their technical expertise, and helps build public confidence in DeFi as a whole.
Developers
Clear guidelines on improving protocol ratings gives developers achievable targets to improve code quality, transparency and documentation, and raises the bar across DeFi.
Auditors
Protocol Reviews incentivize the use of more comprehensive and high-quality audits to achieve a better rating, and disincentivize the use of fake or trivial audits. Additionally, encouraging clear and accessible documentation facilitates auditor mandates.
Protocol Review Process
The Protocol Review process scrutinizes six key elements of deployed smart contracts and rates them according to how closely they follow best practice.
We review the development history of the code in the application and top-level transparency indicators; whether developers are published or anonymous, whether code is public or private, and whether smart contract addresses you interact with are publicized and easily located.
While transparent systems are not guaranteed to be free from exploits, they allow anyone to review them in real time and search for potential problems. This increases the capacity for community monitoring, and is an indicator of how serious a team is about their design process.
High scores are given to projects with clear guidance to smart contracts, public software repositories, actively used code, clear development history and teams with public members.
We verify the quality of system and software documentation, as well as accompanying commentary.
Good documentation indicates time spent on developing protocols to interact with the DeFi ecosystem, allowing other developers to easily integrate their own protocols and effectively interact with the smart contracts themselves. It also proves teams are able to onboard new developers effectively, and demonstrates a commitment to developing effective protocol foundations.
High scores are given to projects with a white paper describing the premise and function of the application, comprehensive software documentation and commentary, and traceability between software documentation and implementation.
We verify the test suite used to validate the application before deployment to the mainnet.
Thorough testing implies that the team has taken the necessary time to develop their product, and demonstrates that code is battle-tested. It proves that code works the way it's supposed to, and ensures against dead code and vulnerabilities. It demonstrates adaptability, and the ability to quickly and effectively implement and validate changes when necessary.
High scores are given to projects with a high test-to-code ratio, with testing covering all the deployed code, instructions for 3rd parties to run tests, reported test results, formal verification and proof of stress testing.
We verify the presence and quality of third-party software audits and bug bounties.
Good security means that protocols are more difficult to hack. Code with a high security score has been audited multiple times, and offers a financial incentive to would-be hackers to submit bug bounties instead.
High scores are given to projects with one or more publicly reported, comprehensive and high-quality audits, as well as a high value bug bounty program.
We verify documentation and disclosure of special admin access controls to the protocol.
Good access controls mean that developers are transparent with the control that they have over the protocol, and demonstrates the upgrade power of a protocol. It also demonstrates that the protocol is able to pause their contracts if problems occur with the contract (hack, bank run, etc).
High scores are given to projects with clear and complete information regarding access controls, and immutable (unchangeable) contracts.
This section goes over the documentation that a protocol may or may not supply about their Oracle usage. Oracles are a fundamental part of DeFi as they are responsible for relaying tons of price data information to thousands of protocols using blockchain technology. Not only are they important for price feeds, but they are also an essential component of transaction verification and security.
Since Oracles play such an important role in the space, it is vital that protocols disclose the complete extent of their Oracle usage. Such information includes the provider, the refresh rate of price feeds, which smart contracts are using Oracles, the Oracle’s key software functions, and more. The availability of this documentation is not only important for developers, but also for users who want to get a good grasp on the robustness of the Oracle used by their favourite protocols. In an era where Oracle manipulations are rampant, it becomes increasingly crucial to prioritize them for the sake of best practices and transparency.
Aerospace
The author of this process has a long history in the avionics industry. Aerospace software development has always maintained that a rigorous and frequently audited software development process leads to safe and secure software that can be supported for many decades. The avionics process is DO-178C. It is significantly more rigorous than the expectations of this review process, but nevertheless informs the overarching philosophy that guides the specification of this review process.
Reviews, Not Audits
What we do is quite different from smart contract auditors in a number of ways. Smart contract auditors are paid by the developers to check their code. The report is written for the developers (who paid for it) and generally written in technical language. The developers set the scope of the audit both in time and subject. The developers decide if they want to publish the report. Effectively, all the power remains in the hands of the developers.
Our target audience is the users of an application. While developers gain value from what we do, we do not ask their permission before doing a review.
Process Quality Reviews
- Evaluate the protocol in aggregate from the perspective of a sophisticated user.
- Focus on adherence to best practice in the development process.
- Measure of Transparency.
- Written for the general public.
- Not funded by the protocol.
- Size and scope determined by a standardized process.
- Review quality of documentation.
- Review quality of code audits.
- Review quality of bug bounties.
- Review quality of testing.
- Valuable in determining the overall trust you could put into the team of a protocol.
- Complementary to software audits.
Code Audits
- Evaluate code vulnerabilities from the perspective of an attacker.
- Focus on the code itself.
- Necessary part of best practice.
- Written for the developer team, NOT the general public.
- Funded by the protocol.
- Developers choose the size and scope of the audit.
- Run automated test tools on the code.
- Very technical content.
- Valuable in determining the security of the reviewed smart contracts.
- Complementary to DeFiSafety Process Quality Reviews.
How to interpret Protocol Reviews
Protocol Reviews are a way of quantifying the degree to which a development team adheres to best practice in the development of their protocol.
High scores indicate that a protocol follows best practice in terms of development, documentation, testing, security and transparency, and can be considered trustworthy and safe because the potential risk vectors are being accounted for and guarded against. It is exceedingly difficult to scam users when your code and access controls are completely public, transparent, tested, and subject to comprehensive audits by reputable firms.
High scores do not indicate that a protocol represents a good financial investment. Instead, they indicate that users of that protocol are much less likely to be scammed or exploited due to oversight or malicious intent.
In contrast, low protocol review scores indicate that users of a protocol may be leaving themselves open to exploitation because some aspects of the protocol deviate from best practice, which may translate into unrecognized vulnerabilities.
Low scores do not indicate a malicious or untrustworthy development team. Instead, they indicate that a protocol has preventable blind spots through which a hack or scam might occur. In theory, a highly trustworthy and exceedingly competent development team could create a protocol with bulletproof code and still receive a low score due to inaccessible documentation and insufficient auditing.
Some examples of what a Protocol score can be used for:
- Use it to judge whether or not you trust a team to survive for the next year.
- Use it to estimate how much effort a developer put into the non-code aspects of their protocol.
- Use it to estimate if a developer will rug-pull their project.