Tag Archives: PHP

First month of GNU Social fixes

In August I began the commit spree on the temporary GNU Social ‘master’ branch over at Gitorious. Temporary because the repository will likely in the future be under the ‘social’ project on that site.

Erkan recently asked for a roadmap of GNU Social development, but unfortunately I’m right now just going to give the next best thing – a summary of what’s been done the latest month. Starting after Evan’s currently last StatusNet commit at Tuesday July 16 2013, 2a70ed2: Merge branch '1.1.x', it was merged into the stone-age stuff of previous GNU Social code from 2010… and new development began. This is the story of our commits from 2013-08-12 to 2013-09-17.

First up, updating some libraries

I guess noone had dared touch the old piece of legacy PHP code that’s called “DB”, i.e. the ancient database abstraction layer StatusNet uses (and doesn’t really get used properly as there are many manual SQL queries in the code still). I didn’t dare do this either, but updates are always nice. So the first essential commit was updating the external library DB to its latest version:

7d8e199 Update to DB_DataObject 1.11.2

There was the idea to migrate everything to MDB2, which would still be able to handle the DataObject classes. However, since it still wouldn’t take care of the biggest problem – functions that are internally called by static methods but aren’t declared static – I figured it wasn’t worth the hassle. And the OpenID plugin supposedly would need lots of work… Anyway, if there’s anything we should do (besides drop PHP!) it’s to migrate over to PDO.

Long-standing merge requests

Then there was work to integrate some fixes from the list of merge requests and such which had not been taken into StatusNet master yet:

3ad3535 Merge commit 'refs/merge-requests/230' of git://gitorious.org/statusnet/mainline into merge-requests/230
ea837ce added missing return statement after showForm call
f433f7c if parameters are not 0, null then limit will be PROFILES_PER_PAGE
56cfd2b comparing a url scheme should be done case insensitively
1095f7a new plugin to check, store and migrate password hashes to crypt()
2e8b729 Issue 3636 request clarity for users without validated emails on instances with RequireValidatedEmail active
7eecd93 _m function for translation seems to be what we use
e47d9ad Added author name to modified file
d6cf6e8 letting the noticeform at the top show, to fix broken reply button javascript
bd60ab2 fix typo on provider_url
f11d157 visual presentation of group's homepage href was its local stream url
20bad68 Added SSL option to web and cli installers
38ac5a7 Automatic memcache support enabler for config
542f00f printf tries to evaluate "%" in paths, echo does not

This felt like a relief for me personally, having written some of those myself. Several other good people had submitted merge requests as well which were taken care of in this round. All the details are in the commit log.

One big problem for new users was also the extreme sluggishness that arises from such a huge, dynamic piece of software as StatusNet. So we automatically enabled Memcache detection if PHP has the module. It requires the administrator to run a local memcache server, but at least no further configuration has to be done after ‘apt-get install memcached’.

What makes GNU Social GNU Social?

And so it became time for a distinction that separates GNU Social from StatusNet somewhat. Something which bothered me in every fresh install I ever made, too:

794163c Default to NOT ask for current location for new users

Privacy. At least we shouldn’t stalk fresh users by asking them where they live everytime they try to post stuff! Another privacy change had already been made (20bad68 regarding SSL-on-install), but this was the first that would immediately affect user experience. SSL improvements were made later as well (commit 81a357e), where a site configured as SSL ‘sometimes’ would never redirect users to http://.

Major overhaul of dataobjects

In a couple of steps to reduce code redundancy and ease future migration to better DB abstraction layers, many of the following commits were directed at the ‘classes/’ directory, finally taking some of the dataobjects to inherit “Managed_DataObject”. Which means that everything from caching to specifying data structure becomes much easier not to say more flexible.

0cb5b6f No need for newline when running 'echo'
3394efc staticGet is a static function
d115cdd Managed_DataObject gets dynamic class detection for staticGet
1a9a8ea staticGet for sub-Managed_DataObject classes now calls parent
6c4c431 Plugins with classes that extend Managed_DataObject get better code reuse
c36608b Some statically called functions in plugins now declared statically
e95f77d Updating all Memcached_DataObject extended classes to Managed_DataObject
2a4dc77 The overloaded DB_DataObject function staticGet is now called getKV
ade2bdc Bookmark class now has schemaDef (for Managed_DataObject)
1710a61 Magicsig class now Managed_DataObject with nicer schemaDef
861e838 pkeyGet is now static and more similar to getKV
3ce5631 Memcached_DataObject::multicache is now properly defined static
7e4718a IMPORTANT - fixed Magicsig to properly overload getKV (prev. staticGet)
761a849 Added shared default plugin list between profiles
923f16a Properly definingStatusNet class static functions with 'static'
97ce71e Managed_DataObject now has listGet for all classes
0785cc2 Don't use DB_DataObject::factory (statically at least)
d5f82bb Class wasn't used anywhere, and file contained wrong class
e40044e Further static declarations of functions
6f4c572 Unnecessary UTF-8 declaration for database
b1465a7 We can now do late static binding (PHP >= 5.3)
3a7261f IMPORTANT: Making prev. Memcached_DataObject working again with schemaDef
66997f2 OStatus classes now has modern schemaDef
0bbcfa7 IMPORTANT - fixed HubSub to properly fetch primary keys
4fab7a9 GNU Social extensions fixes (please read note)
40fe10e Woops, forgot auto_increment (comes with 'serial')
b6cfcfb More info for a proper, fancy-url lighttpd setup

…and during this time the code was broken several times. Part of the fixes also included depending on PHP >= 5.3, which means we can rely on much better object orientation and snazzier functionality (such as late static binding). Unfortunately several problems still exist internally in the DB abstraction layer, where it calls its own non-statically defined functions with static methods. So the ‘PHP Strict Standards’ warning bells keep going off all the time – and still do even after all of the commits in this post.

One important thing that changed during these commits however was the Memcached/Managed_DataObject API, where instead of calling the DB_DataObject function staticGet (which wasn’t static!) it has been renamed getKV (for getKeyValue). Some other related data-fetching functions were also renamed and changed throughout the codebase.

Code reuse is a good thing, viva la OOP!

Then there was a big one. Standardising the way class files are autoloaded, which had already been kinda-maybe-half-made in previous StatusNet code, I went through a lot of the code for plugins and could remove tons of lines of code with statically assigned ‘include_once’ lines. Instead it is all generally handled in the Plugin parent class for the event onAutoload. Many files were moved and renamed in the course of this commit, as they all would fit the same pattern, which can be seen in lib/plugin.php.

de55d8f plugins onAutoload now only overloads if necessary (extlibs etc.)

And then some more cleanup and fixing

More merge requests and general fixes that had been noticed while digging around in the code were implemented.

cb94a29 Syntax error in XMPP config example
fac7371 pivotGet moved into Managed_DataObject
b3e61ce Stronger typing, require array where param array
79e3acf Moved multiGet into Managed_DataObject
f042eea removing empty "planned" GNU Social plugins
07ca304 Changes to GNUsocial plugins
bd24ab6 Fixed comment description in STS plugin
47eb3cf db/core.php lacked User_username (Issue 3299)

And some new parent classes

When working on Free & Social, I rewrote some major parts of the code. A lot of it isn’t quite finished yet (like the theme engine migration), but many bits and pieces can be put back in to GNU Social. Like the fact that just about every Action where one interacts with a form can inherit, and standardise to, a ‘FormAction’ class:

8d57fb7 Added a FormAction extension
cfa699e NewgroupAction converted to extend FormAction
83000f6 Proper definition of $args array in NewgroupAction->prepare
c735a83 Conforming to code layout
0612e5e NewnoticeAction converted to extend FormAction
89b1066 IMPORTANT: parent::handlePost() in NewnoticeAction
e5e3aeb newmessage (and Message class) fixed for FormAction
f0e967f needLogin renamed checkLogin and made a property
b2a0aa2 NewapplicationAction converted to FormAction
c5bf6cb Using a bit more of $this->scoped (Action parent class)

As you can see I migrated a couple of actions as well, but many still need work. And an introduction of a SettingsAction parent class would also be very good!

Tidying up, cleaning and implementing merge request

Back to house cleaning. Some code improvements were made where little effort was needed. Strict typing is very important if one doesn’t enjoy headaches while reading and debugging code. So some of that was introduced (and a little bit more would come later):

747fe9d Tidying up getUser calls to profiles and some events
a9c4bcd Removing unnecessary require_once lines (autoload!)
99312c8 Declaring some more static functions properly
81a357e Putting in functionality so that sites with the "Sometimes" SSL setting allow for users with plugins such as HTTPSEVERYWHERE who wish to use HTTPS to do so without having errors pop up. Specifically this references this issue: http://status.net/open-source/issues/3855#comment-48988.
ade8c69 Twitter cards implementation. Currently only supports 'photo' cards.
f0d86cd Add 'twitter:title' meta tag support.

jbfavre’s Twitter updates were also implemented. I actually haven’t tested these, but jbfavre is a good person whom we can all trust.

The Big JQuery Update

Yes, the web uses Javascript incredibly much today. And unfortunately GNU Social currently even depends on it for full functionality. That’s something I wish to change, to avoid a dependency on automatically executing programs in the browser. But it can undoubtedly increase usability. So updating JQuery to its latest version was next on the todo list:

1eead02 Changed bind() and unbind() to on() and off(). Shortcut for (document).ready().
6fa9062 Changed bind() and live() to on(). Changed .attr() to .prop() for checked and disabled. Shortcut for (document).ready(). This is the first attempt to convert live() elements to on() according to http://api.jquery.com/live/.
af4f2a1 Changed .attr() to .prop() for checked and disabled. Removed "style" removal which I assume was tied to opacity setting on line 9. Replaced "style" setting via attr() on line 12 with css().
1757a65 Shortcut for (document).ready(). Changed .attr() to .prop() for checked and disabled.
3efa107 json2 extlib updated to 2013-05-26 version
a56ad2c Updated jquery extlib to v2.0.3
a4d04d2 Fixed regression from util.js updates + syntax cleanup
3858897 farbtastic removed along with userdesign stuff
dfa1b15 Changing js .live calls to .on for jquery 2.x
438685b jquery javascript was moved to js/extlib
2da9288 jquery-ui updated and moved to js/extlib
56ebe91 jquery form updated and moved to js/extlib
4f065d6 Removed jOverlay as it's outdated and not referenced
0731207 updated jquery-infieldlabel from 0.1.2 to 0.2.1
31bace8 updated and moved jquery-jcrop (no longer .min.js)
3604924 updated and moved jquery-cookie
11f4363 Fixed regression from jquery-cookie update
6de3fc0 jquery 2.x update related fix (.die no longer defined)
5e24600 Minified javascripts are evil! Human readable source, please!
1775fce Added new config for lighttpd.conf that worked on my 1.28 setup.
4822965 Event::handle only takes array $args
83b8523 Events on user registrations now strictly typed
d480ed4 Gravatar pretty much equals disregarding privacy
8935a2f Autocomplete migrated to jquery-ui autocomplete
8140c4f Bad call to joinAdd in Profile.php

It seems to have worked well for me, at least! I believe many of these commits can be imported without any of the previously listed ones, as they should only touch javascript files and some small portions of fairly static code. A modern JQuery library can help a lot to make GNU Social clientside development interesting for more people!

Some of the commits in the latest list are unrelated to JQuery updates (essentially all of the last ones except 8935a2f), but are instead part of tidying up and making the code stricter and less redundant. Many scripts are even thrown away, as they were never referenced anywhere in the code (such as farbtastic and jOverlay).

My favorite commit was “5e24600 Minified javascripts are evil! Human readable source, please!”. In the future we should also have a closer look at LibreJS for compatibility and compliance!

NOTE: A regression was made in jquery 2.x fixes, so please cherry-pick commit f711f9ee7551ac0fe59f1cdc5a375d4825d93e3c too if you’re annoyed with double-submitting favor forms and such ;)

…I leave it to the future to come up with a roadmap for the future :)

Utveckling av “fedsocweb” både stagnerar och frodas

Jag har som bekant ett projekt aktivt med stöd från Internetfonden. Målet är att förbättra den existerande mjukvaran StatusNet, en fri mjukvara för federerade sociala nätverk – eller “federated social web” (fedsocweb). Källkoden jag publicerar går självklart att läsa och projektets resultat går förstås även att använda.

Tillgängliga, liknande mjukvaror

Mycket har hänt sedan jag påbörjade projektet, i detta inlägg ska jag fokusera kring de alternativa mjukvaror som finns inom fedsocweb.

Min utgångspunkt har varit StatusNet, som i mina ögon har en väldigt stor kodbas (228k rader faktiskt kod) som helt enkelt inte är särskilt smidig att uppdatera och bygga vidare på. Det är tydligt att mycket kod hänger kvar “från förr” och orsakar en del kompatibilitetsproblem och huvudvärk när man försöker skapa ny funktionalitet – och inte minst när man försöker standardisera och effektivisera koden.

Sedan november förra året, då jag tog mina första stapplande steg i kodstrukturen för detta stora projekt så har jag försökt bidra med buggfixar och har fått en del av dessa beviljade. Många fler har inte antagits, mestadels eftersom StatusNets huvudutvecklare arbetar på ett annat projekt, Pump.io. Detta nya projekt bygger även det på s.k. “ActivityStreams” och ska implementera OStatus fullt ut, fast i en mindre tungrodd infrastruktur på serversidan. I ett försök att “göra om göra rätt” kanske, men jag har inte hakat på det just det spåret.

Denna stagnation i utvecklingskraft på sociala federerade nätverk är inte unikt för StatusNet. Diaspora* verkar ha fått slut på sitt kapital och riktar in sig på en mer tumblr-remix-aktig satsning än Diaspora-projektets ursprungliga “Facebookdödande” inriktning. Deras nya projekt är helt enkelt nischat för specifika typer av användare och Diaspora* lämnades över “till communityn” i ett mycket ofärdigt skick, om än behändigare kodstruktur än exempelvis StatusNet.

Dessa två plattformer, StatusNet och Diaspora*, har varit de största att dominera sfären av federerad social media. En hittills mer lågmäld spelare har varit Friendica som har varit mer av en satsning på att binda samman separata nätverk för privat bruk än att fungera som en federerad plattform i sig. Sedan StatusNets utveckling stagnerade strax efter sommaren 2011 verkar mycket intresse från bidragande utvecklare ha gått över till Friendica istället. Tyvärr har Friendica en ännu större kodbas än StatusNet (>500k rader kod, mer än dubbelt så stor).

Mycket kod gör projekt jobbiga att sätta sig in i, underhålla och utveckla. Vilket är tre oerhört viktiga egenskaper att bibehålla för projekt med fri mjukvara. Ju högre tröskel in i utvecklingen, desto färre kommer att hjälpa till. Fokus borde alltså ligga på att underhålla lätta, smidiga kodbaser som gör vad de ska på ett enkelt sätt. Mitt mål har därför varit att kapa och meka i StatusNet för att underlätta underhåll. Ett direkt resultat i början var det enkla måttet på antal rader kod men sedan dess även utbyggbarhet och standardisering.

Att mäta resultaten

Jag passade på att nyttja kod-och-projekt-analysatorn på Ohloh för Free & Social, vilket inkluderar all historik från StatusNet (som också finns där). Det syns tydligt när jag började skära som hårdast, i månadsskiftet juni-juli 2012, där 70,000 rader så gott som onödig kod togs bort. Sedan dess har utveckling skett för att separera utseende från funktion, vilket i det stora hela inte lagt till särskilt mycket kod alls. Strax är jag dessutom redo att radera väldigt mycket kod som för tillfället endast ligger kvar som “fallback” om min kod inte täcker upp ett visst användningsområde.

Målet är att kunna byta ut den kod som är skriven i PHP mot något helt valfritt, modul för modul. Det ska bli lättare att köra sin egen StatusNet-instans, eller vilken mjukvara man vill, för att kommunicera federerat med OStatus-protokollet (och även andra förstås).

Det jag inte gjort hittills är att paketera Free & Social-forken för enklare installation på slumpmässiga datorer. Detta kommer när man inte längre behöver paketera hela frontend-motorn med allt vad det innebär för människor som t.ex. vill ha en installation man endast interagerar med via API och tredjepartsklienter.

Projektet tar absolut mer tid än jag hade förväntat mig. Fast då resulterar det även i en mycket större förbättring än vad jag först hade tänkt mig.