before you start writing program, do the following
a. prepare the problem statement
b. make the data flow diagram of the system

after you make the data flow diagram, you will be able to identify what the software needs
next step is
a. design your data base (by normalization)
b. design the data entry screens
c. design the reports
(you will understand the flow your software will have)

finally identify the programming language (i assume you want to do in VB)
so proceed with creating your project, data entry screens, reports and the code.

this way you will follow a scientific way.

All pen drives are essentially do the same purpose.
No one can just be very very reliable. The maximum you can be offered is a replacement of the USB thumb drive. But what about the data.
So before you select any one, make sure the cost is low, and better buy 2 and keep your data duplicated.

Start with what you wish to do.
a. wish to do some programming (then start with C++/Java/.Net) there is a good scope.
b. wish to do some web site design then get some experience on Macromedia/Flash/Adobe
c. with to be database administrator then do some database/oracle/sql
d. wish to be network administrator then do some linux/windows 2003/xp
and so on.

If you just need some basic knowledge then get some books on XP/MS office and start. But what ever you do, suggestion would be to start with formally, at some university, school, institute.