As we move towards the Sport Pilot Regulations, the answer to this question has changed. Under the original ultralight regulations, you may operate 30-minutes before sunrise and 30-minutes after sunset if your aircraft has operating anti-collision lights (strobes).
Once your aircraft is N-numbered, you must follow the regulations that all certified aircraft follow. Unless your aircraft has navigation lights, you may only operate from sunrise to sunset. Navigation lights are the red light on the left side of the aircraft, the green light on the right side of the aircraft and the white light on the rear of the aircraft.
If your aircraft has operating navigation lights, you may operate during 'civil twilight.'
You cannot operate at night unless your aircraft has been approved and certified for night operations by your aircraft manufacturer.