Five Questions to Ask Yourself Before Creating a Web Project

There are a million reasons for tech projects to fail. Miscalculated business models, overestimated demand or bubbling costs, you name it. But over my professional life I’ve seen quite a few projects with brilliant ideas and good potential crumble on seemingly minor errors and oversights. And I think this reason is the bitterest of all, at least for me as a developer. In this article, I want to share my experience and analyze the problems and challenges that backend developers face working on in web applications. I will highlight key points that are often overlooked and explain how to address these obstacles with maximum efficiency. I’m sure this will help you minimize risks and significantly increase your project's chances of success.

No matter how obvious it may sound, this point is crucial: never ever store confidential or sensitive information in your source code. Breaches can lead to financial losses and other serious issues. Sensitive information that should never be stored in code includes:

\

• API keys and access tokens to internal or external services
• Passwords and account data, including database and admin system passwords
• Encryption keys
• Configuration files with sensitive data
• Answers to security questions like mother’s maiden name or pet’s name

\ Instead of storing such information in your project code, use environment variables. For more secure systems, consider using robust secret storage solutions like HashiCorp Vault. AWS Secrets Manager or GitHub secrets can also be useful. The choice of secret storage tools depends on factors such as the project type and size, the team's experience, and the technology stack.

\n If you think storing sensitive information in application code isn’t a serious issue, consider this: in 2022 alone, GitHub detected over 1.7 million potential secrets exposed in public repositories. Imagine how many such pieces of data might exist in private projects where developers are unaware of potential leaks until they occur.

\ Solution: Check your project right now

\ If you already have a project and are now worried about secrets in your code, there are handy solutions that can bring you peace of mind. Manual checks can be time-consuming, so automation is key. Useful tools include:

\

• TruffleHog: Searches for secrets in Git repositories by scanning commit history for API keys and other sensitive data.
• GitLeaks: Detects hard coded secrets like passwords, API keys, and tokens in git repositories.
• GitGuardian: Operates in your local or CI environment to find over 350 types of secrets and other security vulnerabilities.
• GitHub Advanced Security: Automatically scans repositories for known types of secrets and alerts you if any are found.

\ These tools are just a few examples; other popular options include SonarQube and Checkmarx. Both offer paid and free solutions to meet various needs and budgets. The goal here is not to list every tool available, but to highlight the existence of these problems and potential solutions. Recognizing the problem is half the battle. Now, it’s important to allocate time to address it and choose the appropriate tools for your needs.

Surprisingly, this topic is rarely discussed, and some developers are not even aware that using third-party solutions can lead to legal issues and significant problems for their company. Don't believe me? Imagine this scenario: a developer in a small company includes a library distributed under the GNU Affero General Public License (AGPL) in a commercial web product. As a copyleft license, AGPL requires that any software using code released under it be distributed under the same terms. This means all your web application's code, including unique developments, must be open and available for free use and modification. Since our example product is commercial, making its source code available could severely undermine the company's competitive advantage and business model.

\ Serious issues can also arise with projects using libraries with licenses that explicitly prohibit commercial use. And it does not get much better if there is no license at all: in fact, absence of a license poses a significant problem since any code is protected by copyright by default. Licenses grant users the right to use the code under specific conditions, but without a license, there is no legal ground to use the code at all, even if it's publicly accessible.

\ It's worth noting that licensing issues may affect you in different ways depending on your jurisdiction: this matter is particularly relevant for countries that have signed international copyright agreements. For example, the Berne Convention for the Protection of Literary and Artistic Works, one of the main international treaties in this area, currently has around 180 member countries. Therefore, using code without explicit permission would mean violating copyright laws and could lead to legal battles in many places worldwide. However, this doesn't mean you should move to a ‘comfortable’ country just to violate all written and unwritten rules. Let us respect each other, and if someone doesn't want their development used for certain purposes, it's best not to do so even from a human perspective.

\ Solution: Use automated checks and updates

\ As you can see, licensing and copyright issues are complex. To protect yourself and your company in advance, it's best to check the licenses of the libraries and software you use. For libraries, this isn't too difficult; modern package managers already have tools for this. For example, in PHP composer, you can do this with the command `composer licenses`, in Python pip through `pip-licenses`, and in Golang, you can get this information through `go-licenses`.

\ And don’t forget to call these commands when you update dependencies (it’s even better to automate these checks), because the license of a connected library can change in new versions.

In web development, it's common to have multiple versions of a project, such as development (dev), quality assurance (QA), staging, and production. Frequently, I've encountered scenarios where dev/QA and staging versions of a site or web project were accessible to anyone on the Internet. Alarmingly, test versions can sometimes be indexed by search engines more effectively than the primary version, which usually harms the product.

\ The main issue here is that test versions may contain bugs or sensitive, maybe even compromising information. Additionally, beta versions are typically more vulnerable to hacking than the final production. This means that their availability increases the risk of an attacker gaining access to sensitive data, internal code, or even the server itself. This is especially true if you're developing a backend for something like a mobile application, as unauthorized access to test versions of the API can be extremely dangerous.

\ Beyond security risks, duplicate web pages can negatively impact search engine rankings. Search engines like Google may see these duplicates as undesirable content, potentially lowering the ranking of your project's original pages or even removing them from the index altogether.

\ Solution: Devise your security strategy from the very start

\ Don’t skimp on domains. If you need a test version accessible online, purchase a separate domain specifically for it. This simple yet effective measure reduces security risks because attackers will usually check subdomains first. Hosting your test version on any subdomain of the main resource makes it an easy target.

\ Restrict access to all test versions. Ensure that dev, QA, staging, and other versions are not publicly accessible. Configure them to be accessible only through a VPN, for example. This reduces the likelihood of unauthorized access, even if the test domain becomes known to malicious actors.

\ Protect test versions from being indexed. Even if your test versions are only accessible via VPN and hosted on separate secret domains, protect them from search engine indexing using a `robots.txt` file or `noindex` meta tags. This step is crucial, as search engines can sometimes find and index these pages in unexpected ways.

There are security rules that many developers tend to overlook, even though they are critically important and have been established through hard-learned lessons. One such rule is to always hide the real IP address of your project. If the IP address of your servers can be determined through the domain name, it can lead to several issues such as:

\

• DDoS Attacks: Knowing your project's real IP address, attackers can launch a Distributed Denial of Service (DDoS) attack on your server. For instance, a DNS Reflection Amplification attack could overwhelm your server with a massive volume of responses from public DNS servers, making your service unavailable to users and causing significant financial loss.

  \

• Identifying Potential Vulnerabilities: Serious hackers, not just amateurs, can scan open ports and network-exposed software to find and exploit weaknesses. Even well-known services like MongoDB have had significant data breaches due to misconfiguration. Many of these issues could be avoided by simply hiding the real IP address.

\ Solution: Make potential attacker’s life more complicated

\ By concealing your server's real IP address, you make it much harder for attackers to target your system. Using a Content Delivery Network (CDN) or DDoS protection services can be very effective here. Popular options include CloudFlare, which offers both CDN capabilities and DDoS protection for free, as well as services like Imperva (formerly Incapsula) providing similar functions and Qrator, which specializes on protecting Web apps with a Web Application FIrewall, but this may incur costs.

\ While these tools can significantly enhance security, there are additional considerations to keep in mind:

• Email Header IP Leak: If you use your main server for sending emails, the real IP address may be exposed in the email headers, bringing your security efforts down the drain.

  \

• IP History and Whois Requests: Services like DNS History or Whois Request can reveal the historical IP addresses associated with a domain. If your real IP was ever linked to your working domain, you should change it.

  \

• DDoS Protection and API Endpoints: Be cautious when using DDoS protection for domains serving as API endpoints. Protection systems might introduce user verification steps that could disrupt the functioning of your client-side applications by replacing JSON/XML responses with HTML code.

  \

• Outgoing API Requests: When your server sends requests to third-party APIs, it can inadvertently reveal its IP address. Using proxy servers for such requests can help, as changing a proxy is easier than dealing with an attack's aftermath.

\ It's important to remember that hiding your project's real IP address is not a cure-all measure. Closing ports used by your software from the external network is crucial wherever possible. Changing standard ports is a debatable practice; some experts argue it complicates your setup without significant benefits. Often, configuring software to interact via Unix sockets instead of network TCP connections is preferable (if both your project and the software run on the same server). This approach not only increases interaction speed but also enhances security by eliminating the need for open ports.

\ For database management systems (DBMS) or other internal services on separate servers, ensure access is restricted to specific IP addresses that you strictly control. This setup prevents unauthorized access to critical systems, adds an extra security layer, and minimizes the risks of external attacks and data leaks.

This piece of advice is quite straightforward but, yet again, often overlooked: regularly update your project's dependencies and server software. Outdated and vulnerable code is a dream for attackers who can easily exploit it.

\ Solution: Automate your updates

\ You don’t need to update everything manually; many automation tools can help. For example, Dependabot on GitHub automatically detects outdated or vulnerable dependencies and suggests updates.

\ Automating the renewal of security certificates is also crucial. If you use Let's Encrypt certificates, you can automate their renewal with Certbot. Expired certificates can cause some real pain, but automating their renewal is a simple task.

\The same principle applies to server software. If you’re working with Linux, especially Debian/Ubuntu-based distributions, Unattended upgrades can easily handle the task. For larger projects with many servers, tools like Ansible, Chef, Puppet, or Salt are available.

The tips provided here are just a fraction of what backend developers need to remember. I chose to highlight crucial but less frequently discussed topics compared to common ones like SQL injection or CSRF attacks.

\ For a deeper understanding of web application security, consider exploring the OWASP Foundation, a nonprofit organization offering valuable resources. The OWASP Top Ten document, available on their website, lists the most common and critical web application security risks. You'll also find information about lesser-known but equally dangerous attacks.

\ I believe the developer community should be both knowledgeable and supportive in sharing insights and experiences. Therefore, I invite everyone to share their observations and comments, which are valuable to all who work in backend development!