Here are some ways to explore research in math as an undergrad.
There are several opportunities to explore research in pure math as an undergraduate at FSU. One is to do an independent study with a faculty member. This is a private reading course in which you usually explore a specific topic in depth during a semester. Information about undergraduate research can be found through the Center for Undergraduate Research and Academic Engagement. First and second year students can apply for the Undergraduate Research Opportunities Program (UROP), the deadline is usually in August before your second year. In your final year, you may be eligible to participate in the honors in the major program at FSU. This consists of 2-3 semesters of work culminating in an honors thesis project.
You may also wish to consider taking graduate courses, or enrolling in the Combined Bachelor's/Master's Degree Program in Mathematics.
Many colleges and universities have summer REU (Research Experiences for Undergraduates) programs in which you get a stipend to participate in a research project with other students under the direction of a faculty member. Applications for these programs are often due in January and February. The NSF funded programs are listed here, and the AMS has a list here. Some other pages can be found by searching for 'REU math'.
In addition, there are several math specific study abroad programs, where you can spend a semester or a year in in-depth study with other interested students from across the globe. Math in Moscow and the Budapest Semesters in Mathematics are two such well-known programs.
If you're interested in applying to graduate school, many programs require the general and subject GRE tests. You will also need to write a statement of purpose and obtain recommendation letters from 3-4 professors. I recommend talking to a faculty member to compose a short list of schools to apply to. You can find some advice here.
Please let me know of any omissions or errors in the above.