{"id":4852,"date":"2026-08-05T22:19:03","date_gmt":"2026-08-05T16:49:03","guid":{"rendered":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/"},"modified":"2026-08-05T22:19:03","modified_gmt":"2026-08-05T16:49:03","slug":"machine-learning-best-practices-10-tips-for-success-2","status":"publish","type":"post","link":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/","title":{"rendered":"Machine Learning Best Practices: 10 Tips for Success"},"content":{"rendered":"<p>[2023-10-27 03:14:22,891] ERROR: [Worker-7] MainProcess: Uncaught exception: MemoryError<br \/>\n  File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/pandas\/core\/internals\/blocks.py&#8221;, line 393, in grow_labels<br \/>\n    new_labels = np.empty(new_shape, dtype=labels.dtype)<br \/>\nnumpy.core._exceptions._ArrayMemoryError: Unable to allocate 14.2 GiB for an array with shape (1905421, 1000) and data type float64<br \/>\n  File &#8220;\/app\/models\/recommender.py&#8221;, line 142, in predict<br \/>\n    X_transformed = self.pipeline.transform(raw_data)<br \/>\n  File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/pipeline.py&#8221;, line 549, in transform<br \/>\n    Xt = transform.transform(Xt)<br \/>\n  File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/preprocessing\/_encoders.py&#8221;, line 987, in transform<br \/>\n    return self._transform(X, handle_unknown=self.handle_unknown)<br \/>\n  File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/preprocessing\/_encoders.py&#8221;, line 455, in _transform<br \/>\n    X_int, X_mask = self._transform_selected_batch(<br \/>\n  &#8230; [TRUNCATED BY KERNEL OOM KILLER] &#8230;<br \/>\n[2023-10-27 03:14:23,002] CRITICAL: [Kernel] Out of memory: Killed process 142 (python3) total-vm:28452104kB, anon-rss:26142104kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:56212kB oom_score_adj:0<\/p>\n<hr \/>\n<p><strong>TO:<\/strong> Engineering All, Data Science Team, Leadership<br \/>\n<strong>FROM:<\/strong> Senior Site Reliability Engineer (On-Call Rotation 4)<br \/>\n<strong>SUBJECT:<\/strong> POST-MORTEM: Incident #8842 &#8211; The &#8220;Clever&#8221; Model That Ate the Cluster<br \/>\n<strong>STATUS:<\/strong> Resolved (Temporarily, until someone touches the code again)<\/p>\n<p>I\u2019ve been awake for 72 hours. My eyes feel like they\u2019ve been scrubbed with industrial-grade steel wool, and my blood is approximately 40% espresso by volume. While most of you were enjoying your weekend, I was watching our Kubernetes nodes drop like flies because someone thought it was a &#8220;great idea&#8221; to push a 14GB unoptimized feature matrix into a production environment with a 16GB RAM limit.<\/p>\n<p>Let\u2019s be crystal clear: Machine Learning is not magic. It is not a &#8220;vibrant&#8221; new frontier. It is a dangerous, stateful, resource-heavy liability that you have collectively decided to treat like a sandbox. This is a survival guide. Read it, or I will personally revoke your <code>kubectl<\/code> access and make you hand-calculate your gradients on a chalkboard.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_80 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-6a7532370b5bb\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-6a7532370b5bb\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#1_INCIDENT-8842_THE_FALLACY_OF_THE_JUPYTER_NOTEBOOK\" >1. INCIDENT-8842: THE FALLACY OF THE JUPYTER NOTEBOOK<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#2_DEPENDENCY_HELL_IS_A_CHOICE_AND_YOU_CHOSE_WRONG\" >2. DEPENDENCY HELL IS A CHOICE (AND YOU CHOSE WRONG)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#3_SERIALIZATION_PICKLE_IS_A_SECURITY_VULNERABILITY_NOT_A_FORMAT\" >3. SERIALIZATION: PICKLE IS A SECURITY VULNERABILITY, NOT A FORMAT<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#4_THE_LATENCY_TAX_AND_THE_p99_LIE\" >4. THE LATENCY TAX AND THE p99 LIE<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#5_STATEFULNESS_THE_SILENT_KILLER\" >5. STATEFULNESS: THE SILENT KILLER<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#6_MONITORING_IS_NOT_JUST_A_DASHBOARD\" >6. MONITORING IS NOT JUST A DASHBOARD<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#7_THE_%E2%80%9CIDEMPOTENCY%E2%80%9D_PROBLEM\" >7. THE &#8220;IDEMPOTENCY&#8221; PROBLEM<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#THE_%E2%80%9CDEFINITION_OF_DONE%E2%80%9D_CHECKLIST_OR_WHY_YOUR_PR_IS_GETTING_REJECTED\" >THE &#8220;DEFINITION OF DONE&#8221; CHECKLIST (OR: WHY YOUR PR IS GETTING REJECTED)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#Related_Articles\" >Related Articles<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"1_INCIDENT-8842_THE_FALLACY_OF_THE_JUPYTER_NOTEBOOK\"><\/span>1. INCIDENT-8842: THE FALLACY OF THE JUPYTER NOTEBOOK<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The root cause of this weekend\u2019s catastrophe was a fundamental misunderstanding of the difference between a research environment and a production system. In your Jupyter notebooks, you have the luxury of &#8220;restarting the kernel.&#8221; In production, the kernel is the Linux OS, and when it runs out of memory, it doesn&#8217;t give you a polite little red box. It kills the process.<\/p>\n<p>The model in question used <code>pandas==2.1.1<\/code> and <code>scikit-learn==1.3.2<\/code>. The &#8220;clever&#8221; part? A <code>OneHotEncoder<\/code> that wasn&#8217;t configured with <code>sparse_output=True<\/code>. On the training set of 10,000 rows, this worked fine. When it hit the production stream of 2 million concurrent users, the resulting dense matrix attempted to allocate 14.2 GiB of contiguous memory. <\/p>\n<p>Python\u2019s memory management is already a joke, but when you combine it with <code>numpy==1.26.2<\/code>&#8216;s demand for contiguous blocks, you aren&#8217;t just asking for a crash\u2014you&#8217;re demanding one. A notebook is a lie. It\u2019s a curated, stateful snapshot of a moment in time where you ignored technical debt because the &#8220;accuracy&#8221; looked good. Accuracy doesn&#8217;t pay the bills when the p99 latency spikes to 45 seconds because the garbage collector is fighting for its life.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"2_DEPENDENCY_HELL_IS_A_CHOICE_AND_YOU_CHOSE_WRONG\"><\/span>2. DEPENDENCY HELL IS A CHOICE (AND YOU CHOSE WRONG)<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>I looked at the <code>requirements.txt<\/code> for this service. It wasn&#8217;t a requirement file; it was a suicide note. Half the libraries weren&#8217;t pinned, and the other half were pinned to versions that haven&#8217;t been patched since the Obama administration.<\/p>\n<p>Here is a snippet of the <code>pip freeze<\/code> I pulled from the dying container:<\/p>\n<pre class=\"codehilite\"><code class=\"language-text\"># The &quot;I don't care about security or stability&quot; starter pack\nnumpy==1.26.2\npandas==2.1.1\nscikit-learn==1.3.2\nscipy==1.11.3\ntorch==2.1.2+cu121\ntensorflow==2.15.0\nrequests==2.31.0\nurllib3==2.0.7\n# Why is this even here?\nmatplotlib==3.8.2\njupyter-client==8.6.0\n<\/code><\/pre>\n<p>Why is <code>matplotlib<\/code> in a production inference container? Are we plotting graphs for the CPU to look at while it dies? Why is <code>torch<\/code> AND <code>tensorflow<\/code> in the same environment? This is 4GB of dead weight in the container image before we even load a single weight file. <\/p>\n<p>Every unpinned dependency is a ticking time bomb. When <code>pip<\/code> decides to resolve a sub-dependency and pulls in a new version of <code>urllib3<\/code> that breaks your <code>requests<\/code> call, that&#8217;s on you. We use <code>poetry<\/code> or <code>pip-compile<\/code> for a reason. If I see a <code>requirements.txt<\/code> without hashes in the next PR, I\u2019m rejecting it without looking at the code.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"3_SERIALIZATION_PICKLE_IS_A_SECURITY_VULNERABILITY_NOT_A_FORMAT\"><\/span>3. SERIALIZATION: PICKLE IS A SECURITY VULNERABILITY, NOT A FORMAT<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>We found a 2.4GB <code>.pkl<\/code> file in the root directory of the container. Not only is <code>pickle<\/code> notoriously insecure\u2014allowing for arbitrary code execution if someone poisons the model file\u2014but it is also incredibly inefficient for large-scale data. <\/p>\n<p>Worse, you\u2019re tracking this in Git. Git is for code. Git is for text. Git is not for 2GB binary blobs of &#8220;cleverness.&#8221; Every time someone runs <code>git clone<\/code>, they are downloading every version of that model you\u2019ve ever pushed. Our CI\/CD pipeline slowed down by 400% this month because the runner has to pull 15GB of historical garbage just to run a linter.<\/p>\n<p>Use Data Versioning (DVC). Use S3 or an artifact registry. Use <code>joblib<\/code> with compression or, better yet, export to ONNX or TensorRT if you actually care about performance. A <code>.pkl<\/code> file is a lazy solution for a lazy engineer. It couples your model to the specific version of the library that created it. If we upgrade <code>scikit-learn<\/code> from <code>1.3.2<\/code> to <code>1.4.0<\/code>, your pickle might not even load. That\u2019s not a feature; that\u2019s a bug.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"4_THE_LATENCY_TAX_AND_THE_p99_LIE\"><\/span>4. THE LATENCY TAX AND THE p99 LIE<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The &#8220;Data Science&#8221; report said this model had a 98% accuracy. Great. Fantastic. Do you know what the p99 latency was? 1,200ms. <\/p>\n<p>In a microservices architecture, a 1.2-second block is an eternity. It causes upstream connection pooling to exhaust, it triggers retries that lead to a &#8220;thundering herd&#8221; effect, and it eventually brings down the API gateway. <\/p>\n<p>You cannot treat a model like a black box. You need to understand the computational complexity of your <code>transform<\/code> calls.<br \/>\n&#8211; <code>pandas<\/code> operations are single-threaded.<br \/>\n&#8211; The Python Global Interpreter Lock (GIL) is still a thing in <code>3.11<\/code>.<br \/>\n&#8211; Every time you call <code>.apply(lambda x: ...)<\/code> on a DataFrame, a part of my soul dies, and a CPU core sits idle while one core screams.<\/p>\n<p>If your model can&#8217;t return a prediction in under 50ms, it doesn&#8217;t belong in the synchronous request path. Put it in a Celery worker, use a message queue like RabbitMQ, or optimize the math. Stop blaming the &#8220;infrastructure&#8221; for your slow code. The infrastructure is fine; your O(n^2) feature engineering is the problem.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"5_STATEFULNESS_THE_SILENT_KILLER\"><\/span>5. STATEFULNESS: THE SILENT KILLER<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Production systems should be as stateless as possible. Your model, however, is a giant ball of state. You\u2019re loading weights into memory, caching feature lookups in global variables, and\u2014in the most egregious case I saw this weekend\u2014writing temporary files to <code>\/tmp<\/code> without cleaning them up.<\/p>\n<p>The incident was exacerbated because the model was &#8220;warming up&#8221; on startup. It took 4 minutes for the container to become &#8220;Ready.&#8221; In Kubernetes, if your <code>readinessProbe<\/code> fails for 4 minutes, the orchestrator thinks the pod is dead and kills it. Then it starts a new one. Which also takes 4 minutes to warm up. <\/p>\n<p>This is called a &#8220;CrashLoopBackoff.&#8221; We had 50 pods stuck in this loop, each consuming 12GB of RAM during their &#8220;warm-up&#8221; phase, which eventually starved the <code>kubelet<\/code> itself. <\/p>\n<p><strong>Cold Start Requirements:<\/strong><br \/>\n1. Your model should load in under 30 seconds.<br \/>\n2. If it needs more time, you must use a sidecar or a pre-stop\/post-start hook that doesn&#8217;t block the main thread.<br \/>\n3. Use <code>mmap<\/code> for loading weights if possible so the OS can handle page faults instead of you loading the whole thing into the heap.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"6_MONITORING_IS_NOT_JUST_A_DASHBOARD\"><\/span>6. MONITORING IS NOT JUST A DASHBOARD<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>I saw the Grafana dashboard you built. It shows &#8220;Model Accuracy&#8221; and &#8220;Number of Predictions.&#8221; That\u2019s not monitoring; that\u2019s vanity. <\/p>\n<p>Real monitoring for ML in production requires:<br \/>\n&#8211; <strong>Memory RSS vs. Limit:<\/strong> I want to see how close we are to the OOM killer at all times.<br \/>\n&#8211; <strong>Feature Drift:<\/strong> If the input data distribution changes, your model is hallucinating. We need alerts for that.<br \/>\n&#8211; <strong>Inference Latency Percentiles:<\/strong> p50, p90, p99, and p99.9.<br \/>\n&#8211; <strong>Saturation:<\/strong> How many requests are waiting for a thread to become available?<\/p>\n<p>We had no visibility into why the memory was climbing. We just saw the &#8220;Signal 9&#8221; and the silence that followed. We need telemetry inside the code. Use <code>prometheus_client<\/code>. Log the shape of your input tensors. If you\u2019re sending a 1,000-column matrix when the model expects 10, I want to see an error log, not a memory overflow.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"7_THE_%E2%80%9CIDEMPOTENCY%E2%80%9D_PROBLEM\"><\/span>7. THE &#8220;IDEMPOTENCY&#8221; PROBLEM<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>During the recovery, we tried to replay the failed requests. Because your inference logic wasn&#8217;t idempotent\u2014it was updating a &#8220;feature store&#8221; database on every read\u2014we ended up double-counting thousands of events. <\/p>\n<p>A prediction should be a pure function: <code>f(x) = y<\/code>. It should not have side effects. If you need to log the prediction, do it asynchronously. If you need to update a feature, do it in a separate process. Do not mix your &#8220;clever&#8221; math with our system of record.<\/p>\n<hr \/>\n<h2><span class=\"ez-toc-section\" id=\"THE_%E2%80%9CDEFINITION_OF_DONE%E2%80%9D_CHECKLIST_OR_WHY_YOUR_PR_IS_GETTING_REJECTED\"><\/span>THE &#8220;DEFINITION OF DONE&#8221; CHECKLIST (OR: WHY YOUR PR IS GETTING REJECTED)<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>If you want to ship a model to my production environment, you will complete this checklist. If any item is unchecked, don&#8217;t even bother tagging me for a review.<\/p>\n<ul>\n<li>[ ] <strong>Environment Isolation:<\/strong> Does the <code>pyproject.toml<\/code> or <code>requirements.txt<\/code> contain ONLY the libraries needed for inference? (No <code>matplotlib<\/code>, <code>pytest<\/code>, or <code>jupyter<\/code>).<\/li>\n<li>[ ] <strong>Memory Profiling:<\/strong> Have you run <code>memory_profiler<\/code> on a production-sized batch? What is the peak RSS?<\/li>\n<li>[ ] <strong>Load Testing:<\/strong> Have you run a <code>Locust<\/code> or <code>k6<\/code> test for 30 minutes at 2x expected peak load?<\/li>\n<li>[ ] <strong>Serialization:<\/strong> Is the model stored in a non-executable format (ONNX, TFLite, or at least a compressed <code>joblib<\/code>)?<\/li>\n<li>[ ] <strong>Artifact Management:<\/strong> Is the model pulled from an S3 bucket at runtime using a versioned hash, rather than being baked into the Git repo?<\/li>\n<li>[ ] <strong>Error Handling:<\/strong> Does the code catch <code>numpy.core._exceptions._ArrayMemoryError<\/code> and return a 503 instead of letting the kernel kill the whole process?<\/li>\n<li>[ ] <strong>Observability:<\/strong> Are there Prometheus metrics for inference time and input feature shapes?<\/li>\n<li>[ ] <strong>Graceful Degradation:<\/strong> If the model service is down, does the application have a fallback (e.g., a simple heuristic or a cached result)?<\/li>\n<li>[ ] <strong>Container Hygiene:<\/strong> Is the final Docker image under 2GB? (Hint: Use multi-stage builds and <code>python:3.11-slim<\/code>).<\/li>\n<li>[ ] <strong>Documentation:<\/strong> Is there a <code>README.md<\/code> that explains what the features are, without using the word &#8220;leverage&#8221; or &#8220;synergy&#8221;?<\/li>\n<\/ul>\n<p>I\u2019m going home now. I\u2019m going to sleep for 14 hours. When I come back, I expect to see a plan for refactoring the <code>recommender<\/code> service. If I see another <code>MemoryError<\/code> in the logs because someone forgot how math works, I\u2019m moving my desk to the basement and changing my name.<\/p>\n<p>Engineering is about constraints. Machine Learning is not an excuse to forget that.<\/p>\n<p>Regards,<\/p>\n<p>The SRE who kept your &#8220;clever&#8221; code alive. <\/p>\n<hr \/>\n<p><strong>APPENDIX A: THE RECOVERY SCRIPT (FOR WHEN YOU INEVITABLY IGNORE THIS)<\/strong><\/p>\n<pre class=\"codehilite\"><code class=\"language-bash\">#!\/bin\/bash\n# Emergency script to clear stuck pods and reset the node pressure\n# Usage: .\/i_ignored_the_sre_and_broke_prod.sh\n\nNAMESPACE=&quot;production-ml&quot;\n\necho &quot;Checking for OOM-killed pods...&quot;\nkubectl get pods -n $NAMESPACE | grep -i &quot;OOMKilled&quot;\n\necho &quot;Scaling down the offender to 0 to save the cluster...&quot;\nkubectl scale deployment recommender-v2 --replicas=0 -n $NAMESPACE\n\necho &quot;Cleaning up orphaned volumes...&quot;\n# [REDACTED: SRE INTERNAL ONLY]\n\necho &quot;Cluster stabilized. Waiting for Data Science to apologize.&quot;\n<\/code><\/pre>\n<p><strong>APPENDIX B: PIP FREEZE COMPARISON<\/strong><\/p>\n<p><strong>What you gave us:<\/strong><br \/>\n<code>pandas==2.1.1<\/code><br \/>\n<code>numpy==1.26.2<\/code><br \/>\n<code>scikit-learn==1.3.2<\/code><br \/>\n<code>torch==2.1.2<\/code><br \/>\n<code>tensorflow==2.15.0<\/code><br \/>\n<code>matplotlib==3.8.2<\/code><br \/>\n<code>scipy==1.11.3<\/code><\/p>\n<p><strong>What you actually needed:<\/strong><br \/>\n<code>pandas==2.1.1<\/code> (with <code>pyarrow<\/code> backend for memory efficiency)<br \/>\n<code>numpy==1.26.2<\/code><br \/>\n<code>scikit-learn==1.3.2<\/code> (only for the <code>Pipeline<\/code> object)<br \/>\n<code>onnxruntime==1.16.3<\/code> (for actual inference)<br \/>\n<code>fastapi==0.104.1<\/code><br \/>\n<code>uvicorn==0.24.0<\/code><\/p>\n<p>Total size difference: <strong>3.4 GB<\/strong>.<br \/>\nTotal memory overhead reduction: <strong>65%<\/strong>.<br \/>\nTotal SRE sanity regained: <strong>Minimal, but it&#8217;s a start.<\/strong><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Related_Articles\"><\/span>Related Articles<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Explore more insights and best practices:<\/p>\n<ul>\n<li><a href=\"https:\/\/itsupportwale.com\/blog\/html-guide\/\">Html Guide<\/a><\/li>\n<li><a href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-models-a-complete-guide-for-beginners\/\">Machine Learning Models A Complete Guide For Beginners<\/a><\/li>\n<li><a href=\"https:\/\/itsupportwale.com\/blog\/install-laravel-on-ubuntu-20-04-lts-with-apache2-and-php-7-4\/\">Install Laravel On Ubuntu 20 04 Lts With Apache2 And Php 7 4<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>[2023-10-27 03:14:22,891] ERROR: [Worker-7] MainProcess: Uncaught exception: MemoryError File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/pandas\/core\/internals\/blocks.py&#8221;, line 393, in grow_labels new_labels = np.empty(new_shape, dtype=labels.dtype) numpy.core._exceptions._ArrayMemoryError: Unable to allocate 14.2 GiB for an array with shape (1905421, 1000) and data type float64 File &#8220;\/app\/models\/recommender.py&#8221;, line 142, in predict X_transformed = self.pipeline.transform(raw_data) File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/pipeline.py&#8221;, line 549, in transform Xt = transform.transform(Xt) File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/preprocessing\/_encoders.py&#8221;, &#8230; <a title=\"Machine Learning Best Practices: 10 Tips for Success\" class=\"read-more\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\" aria-label=\"Read more  on Machine Learning Best Practices: 10 Tips for Success\">Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-4852","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Machine Learning Best Practices: 10 Tips for Success - ITSupportWale<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning Best Practices: 10 Tips for Success - ITSupportWale\" \/>\n<meta property=\"og:description\" content=\"[2023-10-27 03:14:22,891] ERROR: [Worker-7] MainProcess: Uncaught exception: MemoryError File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/pandas\/core\/internals\/blocks.py&#8221;, line 393, in grow_labels new_labels = np.empty(new_shape, dtype=labels.dtype) numpy.core._exceptions._ArrayMemoryError: Unable to allocate 14.2 GiB for an array with shape (1905421, 1000) and data type float64 File &#8220;\/app\/models\/recommender.py&#8221;, line 142, in predict X_transformed = self.pipeline.transform(raw_data) File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/pipeline.py&#8221;, line 549, in transform Xt = transform.transform(Xt) File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/preprocessing\/_encoders.py&#8221;, ... Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\" \/>\n<meta property=\"og:site_name\" content=\"ITSupportWale\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/Itsupportwale-298547177495978\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-05T16:49:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/itsupportwale.com\/blog\/wp-content\/uploads\/2021\/05\/android-chrome-512x512-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"512\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Techie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Techie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\"},\"author\":{\"name\":\"Techie\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/#\/schema\/person\/8c5a2b3d36396e0a8fd91ec8242fd46d\"},\"headline\":\"Machine Learning Best Practices: 10 Tips for Success\",\"datePublished\":\"2026-08-05T16:49:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\"},\"wordCount\":1813,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/#organization\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\",\"url\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\",\"name\":\"Machine Learning Best Practices: 10 Tips for Success - ITSupportWale\",\"isPartOf\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/#website\"},\"datePublished\":\"2026-08-05T16:49:03+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/itsupportwale.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Machine Learning Best Practices: 10 Tips for Success\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/#website\",\"url\":\"https:\/\/itsupportwale.com\/blog\/\",\"name\":\"ITSupportWale\",\"description\":\"Tips, Tricks, Fixed-Errors, Tutorials &amp; Guides\",\"publisher\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/itsupportwale.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/#organization\",\"name\":\"itsupportwale\",\"url\":\"https:\/\/itsupportwale.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/itsupportwale.com\/blog\/wp-content\/uploads\/2023\/09\/cropped-Logo-trans-without-slogan.png\",\"contentUrl\":\"https:\/\/itsupportwale.com\/blog\/wp-content\/uploads\/2023\/09\/cropped-Logo-trans-without-slogan.png\",\"width\":1119,\"height\":144,\"caption\":\"itsupportwale\"},\"image\":{\"@id\":\"https:\/\/itsupportwale.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/Itsupportwale-298547177495978\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/itsupportwale.com\/blog\/#\/schema\/person\/8c5a2b3d36396e0a8fd91ec8242fd46d\",\"name\":\"Techie\",\"sameAs\":[\"https:\/\/itsupportwale.com\",\"iswblogadmin\"],\"url\":\"https:\/\/itsupportwale.com\/blog\/author\/iswblogadmin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning Best Practices: 10 Tips for Success - ITSupportWale","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning Best Practices: 10 Tips for Success - ITSupportWale","og_description":"[2023-10-27 03:14:22,891] ERROR: [Worker-7] MainProcess: Uncaught exception: MemoryError File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/pandas\/core\/internals\/blocks.py&#8221;, line 393, in grow_labels new_labels = np.empty(new_shape, dtype=labels.dtype) numpy.core._exceptions._ArrayMemoryError: Unable to allocate 14.2 GiB for an array with shape (1905421, 1000) and data type float64 File &#8220;\/app\/models\/recommender.py&#8221;, line 142, in predict X_transformed = self.pipeline.transform(raw_data) File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/pipeline.py&#8221;, line 549, in transform Xt = transform.transform(Xt) File &#8220;\/usr\/local\/lib\/python3.11\/site-packages\/sklearn\/preprocessing\/_encoders.py&#8221;, ... Read more","og_url":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/","og_site_name":"ITSupportWale","article_publisher":"https:\/\/www.facebook.com\/Itsupportwale-298547177495978","article_published_time":"2026-08-05T16:49:03+00:00","og_image":[{"width":512,"height":512,"url":"https:\/\/itsupportwale.com\/blog\/wp-content\/uploads\/2021\/05\/android-chrome-512x512-1.png","type":"image\/png"}],"author":"Techie","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Techie","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#article","isPartOf":{"@id":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/"},"author":{"name":"Techie","@id":"https:\/\/itsupportwale.com\/blog\/#\/schema\/person\/8c5a2b3d36396e0a8fd91ec8242fd46d"},"headline":"Machine Learning Best Practices: 10 Tips for Success","datePublished":"2026-08-05T16:49:03+00:00","mainEntityOfPage":{"@id":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/"},"wordCount":1813,"commentCount":0,"publisher":{"@id":"https:\/\/itsupportwale.com\/blog\/#organization"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/","url":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/","name":"Machine Learning Best Practices: 10 Tips for Success - ITSupportWale","isPartOf":{"@id":"https:\/\/itsupportwale.com\/blog\/#website"},"datePublished":"2026-08-05T16:49:03+00:00","breadcrumb":{"@id":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/itsupportwale.com\/blog\/machine-learning-best-practices-10-tips-for-success-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/itsupportwale.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Machine Learning Best Practices: 10 Tips for Success"}]},{"@type":"WebSite","@id":"https:\/\/itsupportwale.com\/blog\/#website","url":"https:\/\/itsupportwale.com\/blog\/","name":"ITSupportWale","description":"Tips, Tricks, Fixed-Errors, Tutorials &amp; Guides","publisher":{"@id":"https:\/\/itsupportwale.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/itsupportwale.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/itsupportwale.com\/blog\/#organization","name":"itsupportwale","url":"https:\/\/itsupportwale.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/itsupportwale.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/itsupportwale.com\/blog\/wp-content\/uploads\/2023\/09\/cropped-Logo-trans-without-slogan.png","contentUrl":"https:\/\/itsupportwale.com\/blog\/wp-content\/uploads\/2023\/09\/cropped-Logo-trans-without-slogan.png","width":1119,"height":144,"caption":"itsupportwale"},"image":{"@id":"https:\/\/itsupportwale.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Itsupportwale-298547177495978"]},{"@type":"Person","@id":"https:\/\/itsupportwale.com\/blog\/#\/schema\/person\/8c5a2b3d36396e0a8fd91ec8242fd46d","name":"Techie","sameAs":["https:\/\/itsupportwale.com","iswblogadmin"],"url":"https:\/\/itsupportwale.com\/blog\/author\/iswblogadmin\/"}]}},"_links":{"self":[{"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/posts\/4852","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/comments?post=4852"}],"version-history":[{"count":0,"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/posts\/4852\/revisions"}],"wp:attachment":[{"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/media?parent=4852"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/categories?post=4852"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itsupportwale.com\/blog\/wp-json\/wp\/v2\/tags?post=4852"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}