{"id":4093,"date":"2023-04-12T06:40:14","date_gmt":"2023-04-12T06:40:14","guid":{"rendered":"https:\/\/blog.osmosys.co\/?p=4093"},"modified":"2024-02-02T04:57:13","modified_gmt":"2024-02-02T04:57:13","slug":"securing-docker-containers-lessons-learned-from-a-data-loss-incident","status":"publish","type":"post","link":"https:\/\/staging.osmosys.co\/uk\/securing-docker-containers-lessons-learned-from-a-data-loss-incident\/","title":{"rendered":"Securing Docker Containers: Lessons Learned from a Data Loss Incident"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_1 counter-hierarchy ez-toc-counter ez-toc-white ez-toc-container-direction\">\r\n<div class=\"ez-toc-title-container\">\r\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\r\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\r\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/staging.osmosys.co\/uk\/securing-docker-containers-lessons-learned-from-a-data-loss-incident\/#Introduction\" >Introduction<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/staging.osmosys.co\/uk\/securing-docker-containers-lessons-learned-from-a-data-loss-incident\/#Investigation\" >Investigation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/staging.osmosys.co\/uk\/securing-docker-containers-lessons-learned-from-a-data-loss-incident\/#Solution\" >Solution<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/staging.osmosys.co\/uk\/securing-docker-containers-lessons-learned-from-a-data-loss-incident\/#References\" >References<\/a><\/li><\/ul><\/nav><\/div>\r\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Introduction\"><\/span>Introduction<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>We had set up <a href=\"https:\/\/www.mongodb.com\/\" target=\"_blank\" rel=\"noopener\">MongoDB<\/a> as a Docker container for a project and hosted it on our infrastructure server. We chose to use a Docker container for ease of deployment and management. However, one morning, we discovered that the <a href=\"https:\/\/www.mongodb.com\/\" target=\"_blank\" rel=\"noopener\">MongoDB<\/a> data was gone, prompting us to investigate the cause.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Investigation\"><\/span>Investigation<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Upon further investigation, we found that the ports exposed from the Docker container to the host were accessible from the world, even when the server was secured with <a href=\"https:\/\/help.ubuntu.com\/community\/UFW\" target=\"_blank\" rel=\"noopener\">UFW<\/a> and configured to deny all incoming connections unless a rule is added. This was a critical security issue, as it meant that anyone could potentially access our MongoDB instance and the sensitive data stored within it.<\/p>\n\n\n\n<p>We discovered this by attempting to connect to the MongoDB instance from a remote machine and were surprised to find that we were able to connect successfully. We also ran <a href=\"https:\/\/www.redhat.com\/sysadmin\/quick-nmap-inventory\" target=\"_blank\" rel=\"noopener\">nmap scans<\/a> multiple times before, but they did not reveal the particular port, which made this issue even more alarming. This is because the nmap by default only scans most popular 1000 ports and mongoDB port isn&#8217;t among that. Here is the faulty config without any host specified which have caused this issue &#8211;<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"yaml\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Other config\n...\nports:\n  # &lt;Port exposed&gt;:&lt;DB port running inside container&gt;\n  - 27017:27017<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Solution\"><\/span>Solution<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>To immediately mitigate this problem, we changed the <code>docker-compose<\/code> configuration to only expose the MongoDB port to the host and not the public. We had to add the localhost address beside the exposed port to achieve this. Which meant that only applications running on the host machine could access the MongoDB instance.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"yaml\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\"># Other config\n...\nports:\n  # &lt;Port exposed&gt;:&lt;DB port running inside container&gt;\n  - 127.0.0.1:27017:27017<\/pre>\n\n\n\n<p>We also recommend taking backups of your volumes frequently and storing them offsite. This is an important practice for any production environment, as it can help prevent data loss in the event of hardware failures, software bugs, or security breaches. You can follow <a href=\"https:\/\/blog.osmosys.co\/2023\/04\/03\/backup-and-restore-of-docker-volumes-a-step-by-step-guide\/\" target=\"_blank\" rel=\"noreferrer noopener\">this article<\/a> written by my colleague for guidance on how to implement a backup strategy for Docker volumes.<\/p>\n\n\n\n<p> <\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"References\"><\/span>References<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/unix.stackexchange.com\/a\/238643\" target=\"_blank\" rel=\"noopener\">https:\/\/unix.stackexchange.com\/a\/238643<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/nullsec.us\/top-1-000-tcp-and-udp-ports-nmap-default\/\" target=\"_blank\" rel=\"noopener\">https:\/\/nullsec.us\/top-1-000-tcp-and-udp-ports-nmap-default\/<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Introduction We had set up MongoDB as a Docker container for a project and hosted it on our infrastructure server. We chose to use a Docker container for ease of deployment and management. However, one morning, we discovered that the MongoDB data was gone, prompting us to investigate the cause. Investigation Upon further investigation, we [&hellip;]<\/p>\n","protected":false},"author":11,"featured_media":4095,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_lmt_disableupdate":"","_lmt_disable":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[37],"tags":[62,65,87],"class_list":["post-4093","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","tag-data-loss","tag-dockers","tag-security"],"modified_by":null,"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/posts\/4093","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/comments?post=4093"}],"version-history":[{"count":0,"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/posts\/4093\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/media?parent=4093"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/categories?post=4093"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staging.osmosys.co\/uk\/wp-json\/wp\/v2\/tags?post=4093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}