Well, there was a time a few years back when we had pretty regular meetups and casual alley cats. I joined that late on and it led into a load of us playing polo on the regular, fixed gear being put aside as we built polo bikes and went to tourneys etc in London. We struggled for players at times and eventually it looks to have died off.
I think a lot of the longer term Brighton forumers have moved on to road bikes or more off-road focused bikes. I do know a good number of people who still ride fixed and there have been various alley-cats of note recently thanks to @africanherbsman and a few others' efforts.
The key here is to organise things and attend when other people organise...otherwise you'll be right, and there's no scene to speak of. The main problems you'll encounter are apathy and divided attention, there's a lot to do and see in Brighton.