Large Language Models (LLMs) excel at complex reasoning tasks, yet their performance hinges on the quality of their prompts and pipeline structures. Manual prompt design, as used in frameworks like DSPy, poses significant limitations: it is time-intensive, demands substantial expertise, and lacks scalability, restricting the widespread use of LLMs across diverse applications. To overcome these challenges, we introduce AutoDSPy, the first framework to fully automate DSPy pipeline construction using reinforcement learning (RL). AutoDSPy leverages an RL-tuned policy network to dynamically select optimal reasoning modules—such as Chain-of-Thought for logical tasks or ReAct for tool integration—along with inputoutput signatures and execution strategies, entirely eliminating the need for manual configuration. Experimental results on the GSM8K and HotPotQA benchmarks demonstrate that AutoDSPy outperforms traditional DSPy baselines, achieving accuracy gains of up to 4.3% while reducing inference time, even with smaller models like GPT-2 (127M). By integrating RL-based automation, AutoDSPy enhances both efficiency and accessibility, simplifying the development of structured, high-performing LLM solutions and enabling scalability across a wide range of tasks