Emre Sokullu

After Ning's decision to discontinue their free networks, nowadays I've seen a lot of blog posts and tweets from community owners, questioning which free platform to go with. Many of them are comparing solutions such as GROU.PS, Grouply, Spruz, SocialGo, Webs and BuddyPress. As the founder and CEO of GROU.PS and an avid community manager myself (http://www.eylem20.org http://www.barcampistanbul.com http://www.socialbeans.org and of course; http://grou.ps/groudotps) I'll do something pretty unusual and join this conversation directly. Here I'll share with you what I think of and know about each of those do-it-yourself social networking players that are being widely mentioned. 

http://grou.ps/yahoogroups.php

Google has unveiled its upcoming browser based operating system, Chrome OS. If you haven't seen it yet, here's a video: (via Webrazzi)

Google has a great marketing campaign that is now on the front page of Google Docs; it's called Google Docs, a Love Letter and here is their video:

Turkiye'deki internet yasaklari sizin de mi caninizi sikmakta.

Things to bing (or google if you want to):
nginx, libevent, memcache, danga interactive, gearman, mogilefs, nosql, cassandra, hive, hadoop, scribe, thrift, maatkit (for mysql), mmm (for mysql)


MySQL useful settings:

query_cache_size = 0 # on master
query_cache_type = 0 # on master
thread_concurrency = 8 # total cores
max_connections = 750 # shouldn’t exceed that
innodb_buffer_pool_size = 10G  # a little less than the total amount

And our typical sysctl additions; as I've promised - the configuration that lets us serve 1PB per month to 3 million unique visitors:

Cok guzel bir konferansti; duzenledigi icin Ozkan Altuner'e tesekkurler:


Yanimda kartvizitlerimi getirmedigim icin iletisim bilgilerimi paylasamadigim herkesi bana email atmaya emre [at] groups-inc.com ve/veya  buraya uye olmaya davet ediyorum (sorulara verdiginiz cevap sadece ben tarafindan goruntulenecek).

After long hours of research and a heated debate at our office, I've finally come to a conclusion on the difference between 2 fairly new  open source projects that are being used by Facebook too; Hive and Cassandra.

Cassandra: (=BigTable=HBase) saves computational power and time (by indexing with a column based approach) but is a resource hog when it comes to storage.
Hive: doesn't consume that much storage space bit responds slowlier.

So how they are in use at Facebook.

As far as we were able to figure out, friend recommendations are produced by Hive. Hive computations are done by cronjobs. Cassandra is in use when immediate responsiveness becomes a requirement. So for example, for the inbox system.

Know more than that? Have any input on how they are being used? Please let me know in the comments.

Check its last modification date, a la filemtime of PHP. How can you do that? Easy:

Here's Dana Oshiro's coverage: http://www.readwriteweb.com/archives/magic_beans_grow_portable_social_networks.php

• MySQL Cluster is not the solution; consider it as a different database and don't forget that it's not very efficient, so ends up costing you much
• Use Memcached
• Your memcache machine should be close to your application/web servers, not the database
  
• Sharding is hard. There are things you can do before sharding. Replicate as much as you need, so that you can split read and write operations. One machine dedicated to write only.
• • Then the most logical thing to do for sharding is horizontal sharding; which means host different tables in different machines. But this is needed only in extreme cases, as you grow like crazy, like grou.ps
  • Before that, you can maximize your write capabilities, you can get a beefy server with 128MB of ram, 4x Hexacore processors. Note that you'll need to use xtradb on such a configuration because regular mysql setup does not scale well to more than 4 cores, instabilities do occur.
  • And even before that, try optimizing your code. 
• You can optimize your queries by enabling logging of the slow queries
  
• For a 16GB of ram, 400 is the maximum number of connections you need to have on the machine.
• If SHOW PROCESSLIST; displays too many Sleeping connections, no problem, don't worry about them.
• You don't recover from table locks with myisam, auto_increment still requires table lock
• Use maatkit tools. mk_audit recommended as a good start
• If you are replicating, you'll need mk_slave_restart at some point, but don't forget that, it's dangerous, can create inconsistencies
• use mk-query-digest to collect information about the incoming mysql queries; then you can optimize them using the EXPLAIN command - tcpdump is a microsecond level alternative for the very same job
  
• take off unused databases and tables
• for a fast updating environment, cronjob mk-query-digest and mk-duplicate-key-checker
• Consider a dual master setup; use mmm (1.x branch is recommended as of this writing) or flipper - good 
• hot backup strategies: lvm_backup, mk-parallel-dump, innobackup,
• do not use set where you can use enum; set is for options that can be chosed more than once
• vmstat 5 5 to see the status of your disk
• even though you don't use myisam, the minimum key_buffer size should be 32MB
• InfoBright is good for analytics tables
• XFS is good as the underlying filesystem of innodb systems.

If you have other tips, please add in the comments; this list will be updated with your name.

Other Sources:
http://20bits.com/articles/10-tips-for-optimizing-mysql-queries-that-dont-suck/
http://provenscaling.com/blog/2008/10/09/introducing-flipper-for-managing-mysql-master-pairs/
http://provenscaling.com/software/flipper/docs/html/
http://www.howtoforge.com/mysql_master_master_replication
http://www.howtoforge.org/mysql_database_replication

Last week, I've informally started an open discussion on this blog, asking "how to keep good people at your company" - The answers started to come in and I got 2 comments so far; both of them suggesting the importance of materialistic satisfaction:

• Hidayet Dogan, one of the best Turkish PHP programmers I've seen so far (based on his resume and projects) dreams of a Google like office environment, full of gadgets and big LCD displays.
• Eren Yagdiran, whose diverse array of interests can be found on his personal blog, says it's all about money.

I agree with both of them. And I'm very happy to announce you guys that our new Istanbul office that will make its debut this August is going to be just like that - satisfying your inner child with a Google-esque environment.

But I think I found the best answer in the book that I've been reading this weekend, Outliners by Malcolm Gladwell. I'll quote 2 of his sentences:

Those three things--autonomy, complexity, and a connection between effort and reward--are, most people agree, the three qualities that work has to have if it is to be satisfying.

Hard work is a prison sentence only if it does not have meaning. Once it does, it becomes the kind of thing that makes you grab your wife around the waist and dance a jig

At GROU.PS we try to foster a corporate culture that provides those 3 things, all at the same time:

• Autonomy: Our management type can be considered democracy under meritrocratic feudalism. There's a loose hierarchy which gives its nodes the freedom to show their creativity without the boundaries of bureaucracy, and the chance to have your own team if you can prove yourself to your "lord". The hierarchy is not fixed, it's dynamic based on your merits.
  
• Complexity: What we're doing here is literally social operating system. If you think it's just a content management system or something like that (and if you're qualified enough) you're welcome to our office to see what kind of things and what kind of an architecture that we're working. I don't claim it's rocket sciences, but it's pretty close.
  
• Connection between Effort and Reward: GROU.PS is a global venture with chances of being acquired or going IPO. And besides the fixed salary, we are committed to give a generous amount of sweat equity and/or stock options to our employees according to their commitment and role in the team.
  

• As you browse through the pages, the screen flicks, and it temporarily displays randomly located black spots - which results in poor user experience.
• There could be a separate button for Kindle Store
• Buttons seem to be low quality
• Packaging was not as good as I was expecting - my anticipation was a true Apple experience, but it was far from that.
• I miss the multi-touch screen of iPhone, it could make note-taking a breeze - which is, I think, a natural component of our reading experience.
• Poor blog reading machine, because no images, no Flash components - which are crucial parts of blogs.
• Can't delete a book that you've purchased (or there's no easy way)
• Chaging pages is slow, once you lose the page, it's so difficult to find wherever you were at

This is not a blog entry, I'm just asking. Feel free to add your thoughts in the comments. But I should write a blog entry about this soon too.

GROU.PS gibi ortaya cikan yeni genc Turk temelli startup'lar beni heyecanlandirmiyor degil. Ancak ben bugun baslasam bir web 2.0 girisimi yapmazdim. Web 2.0 girisimlerinin sayisi ozellikle Yahoo'nun OddPost'u satin almasindan sonra hizla artti; delicious ve Flickr satin alimlariyla beraber doruk noktasina eristi. Bir ara oyle bir boom yasadik ki, YouTube'ler ve ozellikle su reklam endustrisindeki satin alimlar, ardindan down ekonominin tadini tekrar tatmamiz uzun zaman almadi. Ama neyse teknoloji ekonomisinde isler cok fena gitmiyor ve satin alimlar yeniden aktiflesmeye basladi. Yine de web 2.0 o kadar cazip bir alan olmadigini kanitladi gibi.

Ama bence bunun en buyuk sebebi Facebook Platform ve iPhone'un yazilim dunyasina getirdigi yeni boyutlar. Ozellikle de iPhone... Konum temelli servisler....

Ozellikle Facebook ve iPhone uzerindeki oyun dunyasinda buyuk firsatlar var. Genc bir girisimcinin en kolay flip yolu hic suphesiz bu ortamlarda oyun yazmak. Basta Zynga, EA ve ngmoco genc oyun sirketlerini (microstudios) almaya oyle ac ki; http://gamemakers.ngmoco.com/ tek basina yeterli bir ornek olsa gerek.

Ama flip'i birakin - birkac arkadas bir araya gelerek buralarda baska buyuk isler yapmak icin de bircok firsat mevcut.

Su an bildigim birkac iddiali Turk Facebook developer da var - ama onlari, izinlerini aldiktan sonra, baska bir yazinin malzemesi yapabilirim :)

Exciting news for the internet and search industries; Yahoo gives up its search unit and starts using Microsoft's Bing as its main search provider.





I think this does mean efficiency for both parties, and it's beneficial for everyone. As Steve Ballmer suggests, a stronger competitor to Google means advertising dollars to split; because right now, all the advertising dollars on the web are going to Google as the competitors have negligible market share when they are divided. But 30% market share against Google's 70% is a significant enough and respectable number that gain the attention of advertisers all around the world.

OK.. So I'm mostly positive about this deal. But what's the downside? I think there's 1 downside for Yahoo; the integration time is too long - 24 months - and this might cause an alienation in Yahoo's user base. Because probably, with this news, the rumours of Bing's search technology is so strong will spread out everywhere. And just like what happened with Yahoo - Google years ago, the remaining users of Yahoo may switch to Bing as well, as a result of those words. Probably Yahoo's bet here is that, the remaining users are loyal, they haven't switched to Google for the search technology, so they won't switch to Bing as well. We'll see...

And as for Microsoft, as one of the bankers asked, the financial upsides for MSFT side is not as clear as it is for YHOO. But my gut says this deal will take the combined entity's revenues so high that it will be beneficial for both Yahoo and Microsoft.

I'm glad that Yahoo will have more opportunity to focus on their core values. And again, congratulations Jerry Yang for making the right move and not selling Yahoo a year ago - that alone was a very big decision and I believe, people will appreciate it years later.

If your site isbuilt from heavy php files like us, you're probably using apache's prefork model (instead of a mpm such as worker, which would be more efficient but cause stability problems with zend engine) and as your site grows, yu apache will start forking way too many threads.

 

As a result you may end up forcing hte limits of your operating system's default configurations.

 

At Linux, there are 2 important sysctl configurations that can affect the performance of your site in high-load sitatuons; they are: kernel.shmall and kernel.shmmax

 

You can use their current values by typing:

touch /proc/sys/kernel/shmall

and 

touch /proc/sys/kernel/shmmax

 

shmmaz should be around 32MB.

 

The solution is as easy as increasing it, and fortunately  that doesn't requre kernel recompilation.

 

Here is what you should type:

 

echo 134217728 >/proc/sys/kernel/shmmax

aand

echo 134217728 >/proc/sys/kernel/shmall

 

Now both are 128MB and you're all set. You may want to save these changes in /etc/sysconfig.ctl as well and to see all values, type in sysctl -a

 

Hope this helps,

So Zend 5.5.1 doesn't let you in even though you got their license. It says "this license is already in use" and turns you down? The solution is simple; just unplug your computer from the internet at the launch of Zend; or turn on your firewall so that it doesn't let your Zend communicate with outside world. 

The thing is Zend is communicating with a central server and queries if your license is being used by another computer too at the same time. This is probably to prevent piracy. However, it doesn't work so well in my case and you can end up with not being able to use it even while you're using it fairly.

Zend 6? No way.. It's Eclipse, it's slow and most importantly, it's still immature...

at least as of 11/08/2008 

Hi folks, just wanted to share a quick tip I've found while hacking PHP Smarty. First let me state the problem; on grou.ps home page, we allow our visitors to change the language of the site by clicking the flag buttons. What these nice flag buttons do is simply to add a ?lang=X suffix at the end of the page; it's pretty simple as shown in this example: <a xhref="?lang=en" mce_href="?lang=en"><img xsrc="english_flag.gif" mce_src="english_flag.gif" /></a>

 

Now let me tell you how I allowed Smarty to place all existing GET variables into this equation; because if we don't put them, the localization of a page such as: http://grou.ps/login_recovery.php?lang=tr&email=iltates@gmail.com&serial=699786 becomes a mess; Smarty automatically trims it and the link becomes http://grou.ps/login_recovery.php?lang=tr

Drums please, here comes the solution: here is the actual code, just take a look at it, you'll understand:

<a xhref="?lang=en{foreach from=$smarty.get item=g key=k}&{$k}={$g}{/foreach}" style="border:0;"><img xsrc="images/flag_en.gif" mce_src="images/flag_en.gif" alt="EN" width="21" height="21" style="border:0" /></a> 

This part does the magic by running a simple Smarty foreach loop and leveraging Smarty's reserved variables:

{foreach from=$smarty.get item=g key=k}&{$k}={$g}{/foreach}