This study presents a total Lagrangian smoothed particle hydrodynamics (TLSPH) framework for modeling large-strain elastoplasticity, structural damage, and fracture. The total Lagrangian formulation is designed to efficiently handle arbitrary large deformations encountered in practical engineering applications, while ensuring accuracy, robustness, and ease of implementation. To achieve these goals, an efficient displacement-based approach is employed, in which strain measures are derived exclusively from current geometry. Based on the hyperelastic-based plasticity, stress and other thermodynamic variables are computed directly from a strain energy function, eliminating non-physical energy dissipation in the elastic regime. The logarithmic strain is adopted to ensure material objectivity and to simplify the return mapping algorithm. An explicit time integrator is inefficient for stiff industrial materials due to the strict CFL condition. To date, an implicit integration scheme for finite-strain elastoplasticity has not been developed within the SPH community. It is primarily because the non-linearity of the constitutive equation poses challenges for parallel computing. As a partial solution, this study focuses on optimizing particle resolution to improve efficiency. In particular, non-uniform particle spacing is often necessary for high aspect-ratio structures. Anisotropic kernel functions, aligned with aspect ratios of particle spacing, are incorporated into the approximations to maintain proper number of neighboring particles. Notably, the fixed reference frame with constant neighbor list significantly reduces the computational cost associated with anisotropic kernel functions. The proposed model is validated through large-strain benchmark problems, demonstrating its accuracy and efficiency. Another key contribution of this study is the incorporation of structural damage and fracture models for quasi-brittle materials. The damage and fracture models are based on damage mechanics which describes strength degradation of materials as the evolution of damage-related variables. The mesh-free nature of SPH offers significant advantages in modeling fracture, since crack initiation and propagation are captured naturally without requiring explicit crack tracking and opening models. To validate these models, key parameters, including load-displacement diagram, crack propagation pattern, and crack tip speed, are compared with previous studies.