Unify Storage system.

[NuckChorris]

Right now, the storage system is scattered. It provides 3 builtin storage systems, plus the JSON being used for the OAuth token.

I'd like to suggest unification of all the storage systems into a single JSON-backed system (since it's the smallest files and the world seems to have settled on JSON).

Upgrading is the hardest part, but this could be done transparently by having the Read method automatically fall back on the old system and upgrade them to JSON before returning. We could obviously leave $format in, deprecated.

Or we could just add JSON as a fourth format and convert most of Contra to use it. Discussion is required.

If we want to do this, I'll write a patch, add code to this issue, and upgrade to a pull request.

[joshtwo]

I don't really see why we need an entirely new format for everything. The only format anybody really uses is the default one. If you feel like things need to be cleaned up then just leave in the default one and you don't realistically break any backwards compatibility (since nobody ever really used anything except the default format), nor do you force everything stored to only be data that makes sense in JSON. Only working with PHP code? Storing small amounts of simple data? Then you might as well store what's the most simple and the most natural for PHP.

I'd simply suggest that the OAuth token be stored somewhere that makes more sense instead of in some random file that was created specifically for this purpose in a format the bot doesn't use for anything else except that one tiny piece of data. As far as I know that's the result of practically copying and pasting in code Pickley wrote.

[NuckChorris]

Well, I'm just looking at it and for large storage the current serialization is kinda shitty; also probably slow since it has to invoke the PHP parser instead of a JSON parser. I might bench it and see. Alas, the Contra storage system is rarely used of small amounts of simple data. And anyways, I can do the switch without breaking anything; as far as I am aware, nothing uses the more consoles features of the PHP print_r format, especially in Contra.

[DivinityArcane]

People copy over their old storage folders, and also, the not auto updates. It'd break lots of bots if done it of the blue.

--- Original Message ---

From: "Peter Lejeck" notifications@github.com
Sent: July 19, 2013 1:25 PM
To: "DeathShadow/Contra" Contra@noreply.github.com
Subject: Re: [Contra] Unify Storage system. (#53)

Well, I'm just looking at it and for large storage the current serialization is kinda shitty; also probably slow since it has to invoke the PHP parser instead of a JSON parser. I might bench it and see. Alas, the Contra storage system is rarely used of small amounts of simple data. And anyways, I can do the switch without breaking anything; as far as I am aware, nothing uses the more consoles features of the PHP print_r format, especially in Contra.

---

Reply to this email directly or view it on GitHub:
#53 (comment)

[NuckChorris]

Once again you ignore the possibility of conversion. It's as simple as leaving the code for loading and changing the code for saving; it's not really complicated, and it'd allow other things outside PHP to parse our data (for web admin panels etc.) so I can see substantial benefit.
—
Sent from Mailbox for iPhone

On Fri, Jul 19, 2013 at 6:58 PM, Justin Eittreim notifications@github.com
wrote:

People copy over their old storage folders, and also, the not auto updates. It'd break lots of bots if done it of the blue.
--- Original Message ---
From: "Peter Lejeck" notifications@github.com
Sent: July 19, 2013 1:25 PM
To: "DeathShadow/Contra" Contra@noreply.github.com
Subject: Re: [Contra] Unify Storage system. (#53)

Well, I'm just looking at it and for large storage the current serialization is kinda shitty; also probably slow since it has to invoke the PHP parser instead of a JSON parser. I might bench it and see. Alas, the Contra storage system is rarely used of small amounts of simple data. And anyways, I can do the switch without breaking anything; as far as I am aware, nothing uses the more consoles features of the PHP print_r format, especially in Contra.

Reply to this email directly or view it on GitHub:

#53 (comment)

Reply to this email directly or view it on GitHub:
#53 (comment)

[DivinityArcane]

I realize that. You would need two load methods (or a case with detection) to pull that off. Also, arguably, most web panels and company are php. Yes, it's possible. Weigh the pros and cons with death, since it's basically him and I working on contra.

--- Original Message ---

From: "Peter Lejeck" notifications@github.com
Sent: July 19, 2013 8:04 PM
To: "DeathShadow/Contra" Contra@noreply.github.com
Cc: "Justin Eittreim" eittreim.justin@live.com
Subject: Re: [Contra] Unify Storage system. (#53)

Once again you ignore the possibility of conversion. It's as simple as leaving the code for loading and changing the code for saving; it's not really complicated, and it'd allow other things outside PHP to parse our data (for web admin panels etc.) so I can see substantial benefit.
—
Sent from Mailbox for iPhone

On Fri, Jul 19, 2013 at 6:58 PM, Justin Eittreim notifications@github.com
wrote:

People copy over their old storage folders, and also, the not auto updates. It'd break lots of bots if done it of the blue.
--- Original Message ---
From: "Peter Lejeck" notifications@github.com
Sent: July 19, 2013 1:25 PM
To: "DeathShadow/Contra" Contra@noreply.github.com
Subject: Re: [Contra] Unify Storage system. (#53)

Well, I'm just looking at it and for large storage the current serialization is kinda shitty; also probably slow since it has to invoke the PHP parser instead of a JSON parser. I might bench it and see. Alas, the Contra storage system is rarely used of small amounts of simple data. And anyways, I can do the switch without breaking anything; as far as I am aware, nothing uses the more consoles features of the PHP print_r format, especially in Contra.

Reply to this email directly or view it on GitHub:

#53 (comment)

Reply to this email directly or view it on GitHub:
#53 (comment)

---

Reply to this email directly or view it on GitHub:
#53 (comment)

[NuckChorris]

Presumably death's getting these emails so it'd be nice if he'd weigh in (hint hint) but the case of detection is damn easy (attempt JSON, if failed, attempt BSV). As for most panels being PHP, thats really only true in the PHP community; lots of people use things like RoR or Perl or Python (arguably a collectively larger market than PHP). I had to implement a PHP print_r parser in JS a while back for a bot game leader board, and i can say it's not fun.

[DivinityArcane]

Well, like I said, it's certainly possible [and shouldn't be too hard.] I never liked the idea of storing data using PHP data files, but it's a popular way of doing it in PHP projects (not like PHP bots are by any means ethical, leaving out the fact that the lack of threading makes them horrible.)
JSON would be nice. It's ultimately what I ended up going with for the configuration files in lulzBot (sans the bot config, which is a custom binary format.)

Date: Fri, 19 Jul 2013 22:02:08 -0700
From: notifications@github.com
To: Contra@noreply.github.com
CC: eittreim.justin@live.com
Subject: Re: [Contra] Unify Storage system. (#53)

Presumably death's getting these emails so it'd be nice if he'd weigh in (hint hint) but the case of detection is damn easy (attempt JSON, if failed, attempt BSV). As for most panels being PHP, thats really only true in the PHP community; lots of people use things like RoR or Perl or Python (arguably a collectively larger market than PHP). I had to implement a PHP print_r parser in JS a while back for a bot game leader board, and i can say it's not fun.

—
Reply to this email directly or view it on GitHub.

[NuckChorris]

PHP Bots are horribly inethical to everybody involved, but I really can't imagine any sane PHP project would seriously do this shit with print_r -- it's just that big a hack.

Ping ping @DeathShadow what say you?

[DeathShadow]

go for it.

[joshtwo]

I suppose you do have a point when it comes to integrating Contra save data with other programs and services. I admittedly didn't think of that first.