General Development

Homemade File Formats – bwoogie

bwoogie

Member

Posts: 380
From: kansas usa
Registered: 03-12-2005
Just curious on how you guys create homemade file types, like ... how do you do it? I'm just learning how to make my own format myself.
I basically have an INI file that I have encrypted.
Basically looks like this:

---
[HEADER]
id=file type info;
fileversion=1;
appversion=1;

[OTHER STUFF]
...
etc...
---

For the app that I'm writing, this is fine, but I'd really like more information on how a file should be structured and whatnot.

------------------
~~~boogie woogie woogie~~~

CPUFreak91

Member

Posts: 2337
From:
Registered: 02-01-2005
quote:
Originally posted by bwoogie:
Just curious on how you guys create homemade file types, like ... how do you do it?

It's quite simple really. You decide what you need to put into it. Then if it needs to be a binary file or encrypted you can use system libraries to read/write to the file.

In Bible Dave we use the .lev file type to tell the engine what other files to load. If we wanted to get schnazzy we could make the .lev a .zip archive (just changing the extension) with all the other files added in to it.

For the Pyweek4 game our levels were given a .dat extension but we just used the cPickle library which read/wrote a bunch of lists/arrays, and had it save in an special ASCII format rather than a compressed (or uncompressed) binary. The pickler lets us save variables to files. That way we could assign a variable to the open file instance and continue with normal coding life.

Your encrypted INI file looks useful. I might fiddle around with that kind of stuff myself.

------------------
All Your Base Are Belong To Us!!! chown -r us ./base
"After three days without programming, life becomes meaningless.'' -- Tao of Programming Book 2

"Oh, bother," said the Borg. "We've assimilated Pooh."

"Socialism works great... if there are no people involved." -- Pastor David Ginter, Union Church of Guatemala.

My Programming and Hacker/Geek related Blog

bwoogie

Member

Posts: 380
From: kansas usa
Registered: 03-12-2005
BTW, here is a link that I was following to get my file going
http://www.vbforums.com/showthread.php?t=345097
and here is what I used for encryption
http://www.devarticles.com/c/a/VB.Net/String-Encryption-With-Visual-Basic-.NET/3/

------------------
~~~boogie woogie woogie~~~

Mack

Administrator

Posts: 2779
From:
Registered: 01-20-2001
quote:
Homemade File Formats

Just like mom used to make

------------------

kenman

Member

Posts: 518
From: Janesville WI
Registered: 08-31-2006
I have created several, and create them differently for what their use is. Depending on the use, depends on how I encrypt them. For example temporary files, most likely are going to be set text, but information files are differnt.

Make sure you submit your file extension to filext? i think, can't find it now, it will get you some good redirection from the search engines.

samw3

Member

Posts: 542
From: Toccoa, GA, USA
Registered: 08-15-2006
If you are using a lang like java that can serialize. Just create your objects by hand and serialize them to a file. When you need to load them back, use the deserialize functions and you have the object built and ready to use, no parsing require. However, its not easy to tweak the files by hand.

------------------
Sam Washburn

steveth45

Member

Posts: 536
From: Eugene, OR, USA
Registered: 08-10-2005
quote:
Originally posted by bwoogie:
Just curious on how you guys create homemade file types, like ... how do you do it?


Generally, I find it vastly simpler, and easier to use existing file formats. XML is nice because it human and machine readable and editable and can basically hold almost any linear or hierarchical data. If you are storing images or something with defined formats available, it is simpler to use those.
If you want to store a bunch of chucks of random binary data in a file, you can use a simple id-length scheme and dump the data directly to a file. Example: the first four bytes would be an id representing what type of data, the second four would be the length of the data chunk in bytes, and then the following data in the file would be the data itself. Using this simple system, you can store multiple chunks of unrelated data in the same file.
If you want to create a specific, reusable and useful file format, you should write up a spec, outlining the file format and create example code for loading and saving that particular format. If the file is binary encoded, it should generally specify a particular extension, and the first few bytes should always be the same, like 0xDEADBEEF, or something like that, which can be used to verify that the file is the suggested type. The code that reads in the file should be graceful enough to handle garbage files without crashing or executing arbitrary code. There's no point in registering your file extension unless hundreds of thousands of people are using it.

------------------
+---------+
|steveth45|
+---------+

CPUFreak91

Member

Posts: 2337
From:
Registered: 02-01-2005
quote:
Originally posted by steveth45:
Generally, I find it vastly simpler, and easier to use existing file formats. XML is nice because it human and machine readable and editable and can basically hold almost any linear or hierarchical data.


Haven't had a need for XML yet, but I know I will use them. I can sum my stance on file formats in 5 words:
Bash and XML are KISS

Of course there are certain cases when it's not possible to use either format, but generally I use bash syntax for ASCII data in files.

quote:

There's no point in registering your file extension unless hundreds of thousands of people are using it.


I don't understand that either. Besides, some file types don't even have extensions. Take executable unix files. 95% of the compiled executable Unix files (On OSX, Linux, *BSD, and Solaris) have no extension yet many use the a.out or MISC filetype.

------------------
All Your Base Are Belong To Us!!! chown -r us ./base
"After three days without programming, life becomes meaningless.'' -- Tao of Programming Book 2

"Oh, bother," said the Borg. "We've assimilated Pooh."

"Socialism works great... if there are no people involved." -- Pastor David Ginter, Union Church of Guatemala.

My Programming and Hacker/Geek related Blog

[This message has been edited by CPUFreak91 (edited June 15, 2007).]

Matt Langley
Member

Posts: 247
From: Eugene, OR, USA
Registered: 08-31-2006
quote:
Generally, I find it vastly simpler, and easier to use existing file formats. XML is nice because it human and machine readable and editable and can basically hold almost any linear or hierarchical data.

I was basically going to respond saying the exact same thing, though it's already been said

XML is a great format, very simple and effective. Though keep in mind it can be quite a hefty format so depending on your goal and needs it may not be optimal.

------------------
Matthew Langley
Lead Documentation Engineer
GarageGames