Particle methods and artificial neural networks (ANNs) share striking similarities in their underlying algorithms, both relying on discrete entities that interact and evolve over time. In this work, we introduce two approaches that build on this similarity: the particle-neuron duality and the minimalistic method. The particle-neuron duality provides a framework in which computational particles and artificial neurons are unified into a single mathematical object, the particle-neuron. This allows for the creation of self-learning, physics-compliant models that adapt to complex systems. One successful application of particle-neuron duals is in modelling human organs, where they effectively represent both physical processes and neural activity in the nervous system. The minimalistic method, on the other hand, reduces the complexity of physics-informed machine learning by using neighbour lists as physics-optimized transformer layers. This approach ensures that fundamental physical principles are inherently satisfied while significantly lowering the dimensionality of the problem. By applying these methods to inverse problems in molecular dynamics, fluid dynamics, and granular mechanics, we demonstrate their ability to generalize across different geometries and boundary conditions, even with limited training data. This work opens new possibilities for integrating physics-based models with machine learning, providing a powerful toolkit for solving complex problems in computational science and engineering.