You will need to set up a local development environment with a web server, PHP and MySQL, however there are many ways of achieving this. XAMPP is quite popular, and it works on Windows, Mac, and Linux. There are a lot of alternatives such as MAMP, WAMP, EasyPHP, and Webmatrix.
For Joomla! 3, you should make sure that whichever you choose has PHP Version 5.3.1 or greater, and has MySQL Version 5.1 or greater. Apache web server is most commonly included in these tools, but you can also use an alternative web server such as IIS.
You can also just work from a live web server, but local development environment is much faster and more convenient, and there is no risk of breaking anything on a live site. Most of these software packages such as XAMPP also include a tool to manage your MySQL database called phpMyAdmin.
To test your code, you need a web browser like Firefox or Chrome.
When developing Joomla! extensions, you need a text editor or an Integrated Development Environment (IDE) to edit the source code. If you are just starting out, you can try a simple text editor such as Notepad++ or Textedit. The main requirements are that the text editor should support the UTF-8 format and should display line numbers. If you plan to do a lot of development, then an IDE is a must. The most popular for Joomla are Eclipse and NetBeans.