talks — jd:/dev/blog

Discovering the Tech Community in Toulouse: A Three-Year Journey

Tue, 25 Jun 2024 00:00:00 GMT

When I moved to Toulouse three years ago, I knew almost no one. My only connection was my cofounder, Mehdi. It was an intimidating start, but thanks to the magic of LinkedIn, my presence in the city didn't go unnoticed. Soon, people reached out to me, giving me my first glimpse into the tech scene here. Among those early connections were Denis and Cédric, whose introductions helped bootstrap my network at an impressive speed.

In case you have never heard of Toulouse, it’s the fourth largest city in France (soon to be third), with 0.5 million inhabitants, and the home of Airbus.

Mergify is a remote-first company, so we don’t have an office where I can commute every day and rely on colleagues to easily have a social network. That forced me to go out and expand my social network out in the unknown.

In 2022, I decided to reboot the Toulouse Python Meetup. Despite having over 800 members in the group at the time, the event had gone dormant since COVID 19. I reached out to the previous (idle) organizers, and with the support of Hugo at Mergify, we organized our first meetup. (I wrote more about attending conferences and how my approach changed over the years.) I prepared a quick return of experience talk, which I presented to the three attendees who showed up (!) how we deployed mypy at Mergify.

It was a humble beginning, but it was a start. Reviving in-person events was still challenging, but we persisted, organizing more meetups until the community began to grow. This effort eventually led us to Wannes, who now runs the events, allowing me to focus on other commitments. It’s tough to get people to attend events nowadays, possibly due to the lingering effects of COVID and the convenience of YouTube.

Joining the Tech.Rocks a couple of years ago was another significant milestone. Through this community, I discovered a substantial number of tech professionals in and around Toulouse. Remote work has made it possible for people to work far from their office, and I met people working at companies such as Datadog, Elastic, Spotify, OVHcloud, Ankorstore, AWS, Scaleway, MongoDB, ManoMano, Malt, Zenchef, GitLab — and there are many others I have yet to meet. I regularly organize lunches and dinners with tech folks from this community to foster stronger bonds, which has been incredibly rewarding.

I've had the pleasure of meeting remarkable people, from experienced engineers to startup founders. Toulouse boasts a vibrant ecosystem, smaller than Paris but thriving nonetheless. While it's easy to joke about the tech economy here being heavily dependent on Airbus and its providers, that's not entirely true. There's a diverse range of companies, though the prevalence of service companies (ESNs) in France is notable, with a scarcity of product-based firms.

One of the highlights of the tech calendar here is DevFest, an annual conference organized by the Google Developers Group (GDG). It's one of the most active meetups, although the connection to Google isn't always clear. Nonetheless, it's a fantastic event that brings the community together.

As a business angel, I've had the opportunity to meet incredible founders and teams from startups like Munity, Kotzilla, Roundtable, and more. It's inspiring to see startups in Toulouse that aren't solely focused on aerospace or IoT. There's a budding entrepreneurial spirit here that’s encouraging to witness.

Finally, I must mention IoT Valley — it took me three years to understand what was that IoT Valley I heard about all the time. Originally centered around SigFox, the community has evolved and now encompasses a broader range of startups. I recently had the chance to host a dinner and podcast episode there, giving me a deeper understanding of its scope. Located near Toulouse in Labège, IoT Valley houses numerous companies, many still focused on IoT, but expanding into other areas as well. They might benefit from a rebrand, but marketing isn't typically a strong suit in French tech. Sharing my experiences of building Mergify and working as an entrepreneur and business angel was a highlight, and I look forward to the podcast episode going live.

IoT Valley Founder Dinner

You can listen to the podcast I recorded here.

<iframe style="border-radius:12px" src="https://widget.ausha.co/index.html?podcastId=yX4JgC5WWna7&color=%23ffffff&v=3" width="100%" frameborder="0" scrolling="no"></iframe>

In conclusion, Toulouse's tech scene is dynamic and growing. For anyone considering a move to this city, you'll find a community that's welcoming, innovative, and full of opportunities. Whether you're looking to network, learn, or launch your next venture, Toulouse has something to offer.

And if you move there, send me a mail!

Sponsoring Conferences

Thu, 06 Jun 2024 00:00:00 GMT

Last week, I wrote about my experience attending conferences.

Over the last year, we've tried to expose Mergify at conferences to reach out to developers. We’ve done various conferences in Europe and the US—the largest being QCon San Francisco 2023 and Devoxx France 2024. We sponsored those events and ran booths for several days all day long to talk to engineers.

Mergify booth at QCon San Francisco 2023

The pattern we’ve seen has been interesting. First, QCon San Francisco was the smallest it’s been over the last few years, as far as I can tell. While 1,400 people were expected, counting the number of people seated in the keynote session revealed less than half of that were present. We talked to tens of engineers without great success. It turns out that trying to sell your tool for an early startup like Mergify is not efficient at all in such a place. Companies tend to do that when they are way larger to raise brand awareness and penetrate the market more efficiently.

At our stage, this was a lot of money spent for barely any gain.

As Mehdi, my cofounder and CTO says, “no great engineer will go to a conference to find the next tool they’ll need.” Indeed, I don’t believe any good engineer should wait 6 months for the next conference they will attend to find a product to their technical problems.

Doing market education, as we tried, over a booth, is utopian. Here’s the typical dialogue that would happen:

– Engineer attending the conference: “Hi! What does Mergify do?”
– Mergify staff: “We offer merge queues for your GitHub repository. Do you know about them?”
– Engineer attending the conference trying not to lose face: “Yeah, for sure!”
– Mergify staff: “Do you use one in your team?”
– Engineer: “No, we don’t need one.”
– Mergify: "How so? You're happy merging outdated PR or running a lot of CI on every PR?"
– Engineer: "We… don't… well… err… what are we talking about exactly?…"

The truth is, 95% of the engineers we talked to have no clue what a merge queue is. Actually, 80% of them don’t know anything about Git besides the basics (i.e., commit and push), and chatting for 10 minutes over a booth is not a good place to educate them.

Speaking at conferences is a way better strategy, as the advent of the Developer Evangelist role has demonstrated over the last years. If well executed, it’s cheaper and can have a far better outcome than sponsoring an event.

You could imagine that sponsoring an event buys you a ticket to speak, but it’s not the case by default. Some conferences allow you to buy speaking time in special, dedicated rooms, for example, but you usually don’t get any special treatment over the regular CfP.

I really need to talk about that CfP game.

Attending Conferences

Tue, 04 Jun 2024 00:00:00 GMT

There’s a lot to say about tech conferences, shows, and exhibits in any form. I’ve been to a few of them over the last decade, and I probably can't count anymore at this point.

I thought it’d be interesting to write a thing or two about them and how my experience changed from my first participation in a gathering of tech people to the most recent one.

Goals

What especially changed for me over the last 15 years is the expectation of conferences.

I remember that the first conferences I went to appeared especially exciting because of the content. I was utterly interested in hearing about new technologies, discovering new features, and learning new practices. My goal as a young software engineer was to become the best at my craft, so I’d go there and attend as many lectures as I could.

Sometimes, that would be very challenging. Take the FOSDEM, one of the largest open-source conferences in Europe, with thousands of geeks attending. I attended FOSDEM numerous times. The buildings they use to run the conference have been the same all along those years, and are known to be way too small to welcome many people to most conferences. There are fantastic talks done by some of the world's greatest hackers, but there are more people waiting in the hall to access the talk than people in the room listening to it.

Regular FOSDEM attendance

You could think that this would ruin the conference for most people, but I think it does not. It shows the other very important aspect of conferences: social interaction.

I know there are so many stereotypes about tech guys not interacting with each other. The truth is, as human beings, we crave social connections, and events are a great source of those. Attending my first OpenStack Summit in 2013 allowed me to meet people I only chatted with over IRC, and it was a real game changer later on.

As my engineering career grew, there was not much interesting to learn from the talks. Many lectures became boring or déjà-vu.

I quickly switched sides and became the one giving talks and sharing knowledge. This was great. It gave me a great sense of recognition, validation, fear, and adrenaline. It’s a significant boost for anyone’s career. It was a game changer for me.

COVID

When COVID happened, everything changed.

I remember receiving a phone call in early March 2020 from Sylvain Zimmer, organizer of the dotConferences. I was booked to speak at dotPy, and my talk on Python performances and profiling was ready to go in a couple of days. I would be live on stage in a Parisian theater in front of hundreds of people. Sylvain explained that something was happening, that they couldn’t risk having this event run, and that they had to cancel.

For the next months, every event was canceled, and people shifted to online, remote work, etc. — you know it all. This broke local communities and the habit of many people going to conferences.

I think this shows overall as there are fewer people going to events today than there used to be. People got used to accessing content over the Internet, webinars bloomed, and since many conferences published their lectures online, interest in traveling to a conference reduced a lot.

I relaunched the Python Toulouse meetup group 18 months ago. There were more than 800 members on that group when we announced that we were scheduling a new session in October 2022—3 years after the last one.

We got only 5 attendees.

Since then, we have continued pushing the event every couple of months, and the event has grown back to more than 40 attendees (and I have stepped down from the organizers).

I think this shows well how bad the COVID hit conferences and meetups in general.

Attending Conferences

As I started running Mergify a few years ago, my expectations of conferences shifted again. As we are building a developer tool, so the developer is now a persona we want to reach to make us aware of what we’re building.

There are two ways of doing that, and most developer-focused companies do one or both:

speak at conferences;
expose at conferences (sponsoring).

I’ll write something about event sponsoring some other time. (Update: I did.)

Winning a slot to speak at conferences is not easy: it requires expertise (we have) and time and focus (we don’t have much). In my case, we encourage folks at Mergify to respond to calls for papers, teach other developers the problems we solve, or share our experience on various topics. This is not always working; unfortunately, we are not experts at playing the CfP game—another topic I should write about.

First, I noticed that while there are more and more software engineers, many of them don’t care about going to conferences. They know most of the talks are already online or will be. As the CfP game is getting professional, many talks you see at conferences have already been lectured, filmed, and published online. Engineers valuing their time might not go to conferences in the end.

Some conferences are trying to fix that by not publishing their talks online. I think it can be a good strategy in certain cases, but as many conferences invite speakers with talks running for months if not years, it’s likely you can already watch the content online anyway.

Second, many events and conferences still overpromise the number of people actually attending. It’s likely the pre-COVID level is not back everywhere.

Last, the average technical level of expertise of both speakers and attendees fluctuates a lot. However, the more I think about it, the less I see a pattern. Some community-run conferences have great and poor content simultaneously; some professional conferences have attendees with low-skill but great speakers. It’s hard to have a rule, and I think it’s really on a case-by-case basis—and it might be subjective, after all.

Those issues might be anecdotic for most, though they’re not for me as they explain partly why Mergify's sponsoring of events has been mostly a failure over the last year.

But I’ll talk about that later.

Being in Charge

Fri, 17 Apr 2020 00:00:00 GMT

If you never heard of the 10x engineer myth, it's a pretty great concept. It boils down to the idea where an engineer could be 10x more efficient than a random engineer. I find this fantastically twisted.

Last week, I sat and chat with Alexis Monville in Le Podcast — a podcast that equips you to make positive change in your organization. We talked about that 10x Engineer myth, and from there we digressed on how to grow your career and handle the different aspects of it.

This was a very interesting exchange. Alexis is actually going to publish a new book next month (May 2020) entitled I am a Software Engineer and I am in charge.

Lucky me, this week, I had the chance to be able to read the book before everybody else — which means I actually read after our recording. I understood why Alexis said that a lot of what I was talking about during our podcast resonated with him. I send a detailed review of the book to Alexis and Michael if you're curious. I'm definitely recommending this book if you want to stop complaining about your job and start understanding how to pull the strings.

I wish I had this book available 10 years ago! 😅

Attending FOSDEM 2020

Thu, 06 Feb 2020 00:00:00 GMT

This weekend, I've been lucky to attend again the FOSDEM conference, one of the largest open-source conference out there.

I had a talk scheduled in the Python devroom on Saturday about building a production-ready profiling in Python. This was a good overview of the work I've been doing at Datadog for the last few months.

The video and slides are available below.

Your browser does not support the video tag.

The talk went well, attended by a few hundred people. I had a few interesting exchanges with people being interested and having some ideas about improvement.

Podcast.init: Gnocchi, a Time Series Database for your Metrics

Tue, 11 Dec 2018 00:00:00 GMT

A few weeks ago, Tobias Macey contacted me as he wanted to talk about Gnocchi, the time series database I've been working on for the last few years.

It was a great opportunity to talk about the project, so I jumped on it! We talk about how Gnocchi came to life, how we built its architecture, the challenges we met, what kind of trade-off we made, etc.

You can list to this episode here.

Attending PyCon FR 2017

Thu, 31 Aug 2017 00:00:00 GMT

The French edition of the annual Python conference, PyCon FR, will happen in Toulouse from 21st to 24th September.

I skipped the last few PyCon FR, but this year I will be back with a one-hour long talk entitled "Scalable and distributed applications in Python". It will take place on Saturday afternoon. I will lay out many topics that will be covered in the book I'm working on, Scaling Python!

The Thursday and Friday will be dedicated to development sprints. I will be there with my friend Mehdi running a session for Gnocchi! We'll spend time teaching new contributors how to use it or how to send love and patches to the project. If you're into Python and want to learn about timeseries management, it's an excellent occasion to join us for some fun. 😎

To join the sprint and the conference, visit the PyCon FR website and register.

OpenStack Summit Boston 2017 recap

Mon, 15 May 2017 00:00:00 GMT

The first OpenStack Summit of 2017 was last week, in Boston, MA, USA. I was able to attend as I've been selected to give 3 talks, to help for a hands-on and to animate an on-boarding session. This made sure I was a bit busy every day, which was good.

This is the first summit to happen since the new Project Team Gathering (PTG) happened last February. I was unable to attend this first PTG back then, as there was no way to justify my presence there. The OpenStack Telemetry team that I lead is pretty small. People don't really need to talk to each other face to face to discuss: therefore we decided to not ask to be present during the last PTG event.

The Telemetry on-boarding session that I organized with my fellow developer Gordon Chung on Tuesday had only 3 people showing up to ask a few questions about Telemetry. The session lasted 15 minutes on 90 planned. We shared that session with CloudKitty, for which nobody showed up for. When you think about it, this was really disappointing but did not come as a surprise.

First, the amount of company engaging developers into OpenStack has shrunk drastically during the last year. Secondly, since there's now another event (the PTG) twice a year, it seems pretty clear that every developer will not be able to attend all the 4 events every year, creating dispersion in the community.

I personally was glad to attend the Summit rather than the PTG, as it is more valuable to meet operators and users than developers to gather feedback. However, meeting everyone at the same time would be great, especially for smaller teams. The PTG scattered some teams to a point that many of developers of those lineups won't go to either the PTG nor the OpenStack. As a consequence, I won't have any meeting point in the future with many of my fellow developers around OpenStack. I warned the Technical Committee last year about this when it was decided to reorganize the events. I'm glad to be right but I'm a bit sad that the Foundation did not listen.

Though all the projects I work on tend to follow the good practice I wrote last year. Therefore I cannot say that it has huge consequences on the projects I work on. It's a loss as it makes it harder to reach users and operators for some of us. It also reduces our occasion for social interaction, which was a great benefit. But it will not prevent us from building great software anyway!

The few other sessions of The Forum (the space dedicated to developers during the Summit) that I attended discussed various technical things, and some sessions were pretty empty. I wonder if it was a lack of interest of people or if people were unable to travel to discuss those items. Anyhow, at this stage I am not sure it would have really mattered: this has been my 9th OpenStack Summit and many of the subjects discussed already have been discussed multiple time with barely any change since. Talk is cheap. Furthermore, most of the discussion were not made by stakeholders of the various projects involved, but by people on the side, or by members of the Technical Committee. There is just unfortunately too much of wishful thinking.

On the talk side, my presentation with Alex Krzos entitled Telemetry and the 10,000 instances went pretty well. We demonstrated what how we tested the performance of the telemetry stack.

Same goes for my hands-on with the CloudKitty developers, where we managed to explain how Ceilometer, Gnocchi, and CloudKitty were able to work with each other to create nice billing reports. The last day was concluded with my talk on collectd and Gnocchi with Emma, which was short and to the point.

My final talk was about the status and roadmap of the OpenStack Telemetry team where I tried to explain how the Telemetry works and what we might do (or not) in the next cycles. It was pretty short as we barely have a roadmap, the project having 3 developers doing 80% of the work.

I was also able to catch up with Nubeliu about their Gnocchi usage. They presented a nice demo of the cloud monitoring solution they build on top of Gnocchi. They completely understood how to use Gnocchi to store a large number of metrics at scale and how to leverage the API to render what's happening in your infrastructure. It is pretty amazing.

While I missed the energy and the drive that the design session used to have in the first summits, it has been a pretty good summit. I was especially happy to be able to discuss OpenStack Telemetry and Gnocchi. The feedback I gathered was tremendous and terrific and I'm looking forward to the work we'll achieve in the next months!

FOSDEM 2017, recap

Mon, 06 Feb 2017 00:00:00 GMT

Last week-end, I was in Brussels, Belgium for the 2017 edition of the FOSDEM, one of the greatest open source developer conference.

This year, I decided to propose a talk about Gnocchi which was accepted in the Python devroom. The track was very well organized (thanks to Stéphane Wirtel) and I was able to present Gnocchi to a room full of Python developers!

I've explained why we created Gnocchi and how we did it, and finally briefly explained how to use it with the command-line interface or in a Python application using the SDK.

You can check the slides below and [the video of the talk (https://video.fosdem.org/2017/UD2.120/storing_metrics_gnocchi.mp4).

Attending OpenStack Summit Ocata

Mon, 31 Oct 2016 00:00:00 GMT

For the last time in 2016, I flew out to the OpenStack Summit in Barcelona, where I had the chance to meet (again) a lot of my fellow OpenStack contributors there.

How To Work Upstream with OpenStack

My week started by giving a talk about How To Work Upstream with OpenStack where I explained, accompanied by Ryota and Ashiq, to the audience how to contribute upstream to OpenStack. It went well and was well received by the public – you can watch the video below or
download the slides.

Python 3 in telemetry projects

I've attended a few interesting cross-project sessions, which helped me getting some prioritization for my work during the next few months.

The Python 3 porting effort is blocked for a while in Nova and Swift for various (mostly non-technical) reasons, while almost all other projects are working correctly. On the other hand, we have committed the telemetry projects to be the first one to drop Python 2 support has soon as it is possible. The next steps are to be sure downstream is ready and enable functional testing in devstack with Python 3.

Ceilometer deprecation

The Ceilometer sessions were really interesting, are we mainly discussed deprecating and removing old crufts that are not or should not be used anymore. The main change will be the depreciation of the Ceilometer API. It has been clear for more than a year that Gnocchi is the way-to-go to store and provide access to metrics, but we failed at announcing wildly. A lot of the people I talked to during the summit were not aware that the Ceilometer API was not a good pick, and that Gnocchi was the now recommended storage backend. Bad communication from our side – but we are going to fix it as of now.

We also committed to simplify the current architecture by removing the collector, which has now be made obsolete by the agent based architecture that was implemented during the last development cycles.

Aodh alarm timeout

We had a feature proposal for a while in Aodh that we postponed for too long already: having timeout triggered after not having seen some events. This seems to be a functionality requested by NFV users – something we want Aodh to cover.

We spent some time discussing this feature, and now that we all have a clear understanding of the use case, we'll work on having a clear path to the implementation.

I've also attended a session with the Vitrage developers in order to discuss how we could work better together, as they rely on Aodh. It seems there might be some convergence in the future, which would be very welcome. Wait'n see.

Gnocchi improvement, past and future

The Gnocchi session ran smoothly, and everyone seemed happy with the work we have done so far. We've made some impressive improvement in Gnocchi 3.0 – as I already covered previously – and Gordon Chung presented a short talk about the performance difference metered while working on this new version of Gnocchi:

The return of the InfluxDB driver is on the table, as Sam Morrison proposed a patch for that while back. While it's not as fast and scalable as other drivers, it offers a good alternative for people having to use it.

Leandro Reox presented how to do capacity planning using Ceilometer and Gnocchi, presenting the projects at the same time:

It is pretty impressive to see what they achieved with this project, and I'm looking forward to being able to check how it works inside.

PTG and beyond

The next meeting is supposed to be the new OpenStack PTG in February in Atlanta, though we did not request any specific space there. While the team love seeing each other face-to-face every few months, we achieved to follow all of the guidelines I listed recently on good open source project management, meaning we are able to work very well asynchronously and remotely. There is no need to put hard requirements on people wanting to participate in our community. Nevertheless, I expect cross-projects discussions that will happen to still concern the OpenStack Telemetry projects.

In the end, we're all very happy with our past and future roadmaps and I'm looking forward to achieving our next big milestones with our amazing telemetry team!

Gnocchi talk at the Paris Monitoring Meetup #6

Fri, 27 May 2016 00:00:00 GMT

Last week was the sixth edition of the Paris Monitoring Meetup, where I was invited as a speaker to present and talk about Gnocchi.

There was around 50 persons in the room, listening to my presentation of Gnocchi.

The talk went fine and I had a few interesting questions and feedback. One interesting point that keeps coming when talking about Gnocchi, is its OpenStack label, which scares away a lot of people. We definitely need to continue explaining that the project work stand-alone has a no dependency on OpenStack, just a great integration with it.

The Monitoring-fr organization also interviewed me after the meetup about Gnocchi. The interview is in French, obviously. I talk about Gnocchi, what it does, how it does it and why we started the project a couple of years ago. Enjoy, and let me know what you think!

FOSDEM 2016, recap

Sat, 06 Feb 2016 00:00:00 GMT

Last week-end, I was in Brussels, Belgium for the FOSDEM, one of the greatest open source developer conference. I was not sure to go there this year (I already skipped it in 2015), but it turned out I was requested to do a talk in the shared Lua & GNU Guile devroom.

As a long time Lua user and developer, and a follower of GNU Guile for several years, the organizer asked me to run a talk that would be a link between the two languages.

I've entitled my talk "How awesome ended up with Lua and not Guile" and gave it to a room full of interested users of the awesome window manager 🙂.

We continued with a panel discussion entitled "The future of small languages Experience of Lua and Guile" composed of Andy Wingo, Christopher Webber, Ludovic Courtès, Etiene Dalcol, Hisham Muhammaad and myself. It was a pretty interesting discussion, where both language shared their views on the state of their languages.

It was a bit awkward to talk about Lua & Guile whereas most of my knowledge was years old, but it turns out many things didn't change. I hope I was able to provide interesting hindsight to both community. Finally, it was a pretty interesting FOSDEM to me, and it was a long time I didn't give talk here, so I really enjoyed it. See you next year!

Gnocchi talk at OpenStack Paris Meetup #16

Mon, 05 Oct 2015 00:00:00 GMT

Last week, I've been invited to the OpenStack Paris meetup #16, whose subject was about metrics in OpenStack. Last time I spoke at this meetup was back in 2012, during the OpenStack Paris meetup #2. A very long time ago!

I talked for half an hour about Gnocchi, the OpenStack project I've been running for 18 months now. I started by explaining the story behind the project and why we needed to build it. Ceilometer has an interesting history and had a curious roadmap these last year, and I summarized that briefly. Then I talk about how Gnocchi works and what it offers to users and operators. The slides where full of JSON, but I imagine it offered a interesting view of what the API looks like and how easy it is to operate. This also allowed me to emphasize how many use cases are actually really covered and solved, contrary to what Ceilometer did so far. The talk has been well received and I got a few interesting questions at the end.

OpenStack Design Summit Icehouse, from a Ceilometer point of view

Wed, 13 Nov 2013 00:00:00 GMT

Last week was the OpenStack Design Summit Icehouse in Hong-Kong where we, OpenStack developers, discussed and designed the new OpenStack release (Icehouse) that is coming up.

The week has been wonderful. It was my second OpenStack design summit, and I loved it. Bumping into various people I've never met so far and worked with online was a real pleasure. As it was to meet again with fellow OpenStack developers! The event organisation was great, as were the parties. :-)

On the last day, I had the chance to present a talk with Eoghan Glynn and Nick Barcet how we built the auto-scaling feature in Heat, implementing the "alarming" feature needed in Ceilometer.

Design sessions

This time, Ceilometer design sessions were spread on 3 days. Everything we talked about has its Etherpad instance. The discussions were interesting, and the amount of feedback gathered is big and is going to be very useful.

There's a lot of people and companies using Ceilometer now, and the project is getting more and more traction in general. There's a lot of different way to use it and to bend it to one's needs. Considering the amount of features and options that is provided, building functionality with a genericized approach it making Ceilometer useful for a lot of different and interesting use-cases.

Icehouse roadmap

The list of blueprints targeting Icehouse is available, but not yet complete. I expect people to start filling this list in the next days. If you want to propose blueprints, you're free to do so and inform us about it so we can validate it. The same applies if you wish to implement one of them!

Thereafter, I try to guess what the roadmap will look like in the upcoming weeks for Ceilometer based on the discussion we had last week during the summit.

Events management

A lot of work is going to be put into event management. Ceilometer plans to store notifications sent using oslo.messaging by OpenStack projects. Some work already got merge for Havana, but the API part and future improvements and ideas will continue to flow into the Icehouse release.

Agents and group management

A lot has been discussed around the polling agents and around the alarm evaluator agent.

The current state of the ceilometer-central-agent disallows any kind of high-availability and load-balancing, as the polling task are kept and scheduled on only one node.

The high-availability part is already covered by a custom mechanism built into ceilometer-alarm-evaluator, but it came clear to us that a more generic approach is needed. A lot of other projects needs this kind of functionality, and a pattern have been pointed out. A blueprint about group membership has been discussed in an Oslo session, and will end into a new Python library written to solve this in Ceilometer and in other projects.

TaskFlow will also probably be leveraged to solve the task distribution issue.

Documentation

Since a few weeks, Ceilometer auto-generates its API reference documentation using sphinxcontrib-docbookrestapi that parses our API code that uses WSME.

We also want to start writing a user guide, and we'll do that inside our own repository. That way, I hope that we will be the first project in OpenStack to require documentation to be incorporated into every patch that's being sent to Ceilometer. This is the best way to assure that nothing can be changed nor added without being accompanied with a documentation update.

Tempest testing

Testing of Ceilometer already has been a subject during the previous design summit about testing. We already put a large effort on Tempest testing in this last cycle, but we encountered a lot of small issues that we had to tackle to achieve something. Some Ceilometer basic tests are already on their way into Tempest, so this is something that is going to be achieved very soon.

Ultimately, I would also want Ceilometer moving towards providing its own set of Tempest tests as part of the code base. That way, it'd be as easy for core reviewers to refuse a patch if it doesn't provide functional tests as it is to refuse it if it doesn't provide unit tests. As we'll do for the documentation.

API improvements

Once again, a few API improvements will probably be implemented, like aggregation or the ability to specify multiple queries with OR and AND operators.

Roll-up, archiving of data

There seems to be interest in archiving and rolling-up the data stored by Ceilometer, so work in this area is to be expected. Supporting multiple data storage driver in parallel seems to be something that needs to be done for this and other aspects of Ceilometer feature set.

Alarming

The alarming feature set is already big, and the work that has been accomplished pretty amazing. A few improvements will be made, as retrieving better metrics and building better statistics (exclusion of low quality data points).

OpenStack Design Summit Havana, from a Ceilometer point of view

Thu, 25 Apr 2013 00:00:00 GMT

Last week was the OpenStack Design Summit in Portland, OR where we, developers, discussed and designed the new OpenStack release (Havana) coming up.

The summit has been wonderful. It was my first OpenStack design summit -- even more as a PTL -- and bumping into various people I've never met so far and worked with online only was a real pleasure!

Nick Barcet from eNovance, our dear previous Ceilometer PTL, and myself, talked about Ceilometer and presented the work that has been done for Grizzly, with some previews of what we'll like to see done for its Havana release.

Design sessions

Ceilometer had his design sessions during the last days of the summit. We noted a lot of things and commented during the sessions in our Etherpads instances.

The first session was a description of Ceilometer core architecture for interested people, and was a wonderful success considering that the room was packed. Our Doug Hellmann did a wonderful job introducing people to Ceilometer and answering question.

The next session was about getting feedbacks from our users. We had a lot of surprise to discover wonderful real use-cases and deployments, like the CERN using Ceilometer and generating 2 GB of data per day!

The following sessions ran on Thursday and were much more about new features discussion. A lot ot already existing blueprints were discussed and quickly validated during the first morning session. Then, Sandy Walsh introduced the architecture they use inside StackTach, so we can start thinking about getting things from it into Ceilometer.

API improvements were discussed without surprises and with a good consensus on what needs to be done. The four following sessions that occupied a lot of the days were related to alarming. All were lead by Eoghan Glynn, from Red Hat, who did an amazing job presenting the possible architectures with theirs pros and cons. Actually, all we had to do was to nod to his designs and acknowledge the plan on how to build this.

That last two sessions were about discussing advanced models for billing where we got some interesting feedback from Daniel Dyer from HP, and then were a quick follow-up of the StackTach presentation from the morning session.

Havana roadmap

The list of blueprints targeting Havana is available and should be finished by next week. If you want to propose blueprints, you're free to do so and inform us about it so we can validate it. The same applies if you wish to implement one of them!

API extension

I do think the API version 2 is going to be heavily extended during this release cycle. We need more feature, like the group-by
functionality.

Healthnmon

In parallel of the design sessions, discussions took place in the unconference room with the Healthnmon developers to figure out a plan in order to merge some of their efforts into Ceilometer. They should provide a component to help Ceilometer supports more hypervisors than it currently does.

Alarming

Alarming is definitely going to be the next big project for Ceilometer. Today, Eoghan and I started building blueprints on alarming, centralised in a general blueprint.

We know this is going to happen for real and very soon, thanks to the engagements of eNovance and Red Hat who are committing resources to this amazing project!

Hy, Lisp in Python

Wed, 03 Apr 2013 00:00:00 GMT

I've meant to look at Hy since Paul Tagliamonte started to talk to me about it, but never took a chance until now. Yesterday, Paul indicated it was a good time for me to start looking at it, so I spent a few hours playing.

But what's Hy?

Python is very nice: it has a great community and a wide range of useful libraries. But let's face it, it misses a great language.

Hy is an implementation of a Lisp on top of Python.

Technically, Hy is built directly with a custom made parser (for now) which then translates expressions using the Python AST module to generate code, which is then run by Python. Therefore, it shares the same properties as Python, and is a Lisp-1 (i.e. with a single namespace for symbols and functions).

If you're interested to listen Paul talking about Hy during last PyCon US, I recommend watching his lightning talk. As the name implies, it's only a few minutes long.

Does it work?

I've been cloning the code and played around a bit with Hy. And to my greatest surprise and pleasure, it works quite well. You can imagine writing Python from there easily. Part of the syntax smells like Clojure's, which looks like a good thing since they're playing in the same area.

You can try a Hy REPL in your Web browser if you want.

Here's what some code look like:

(import requests)

(setv req (requests.get "http://hy.pault.ag"))
(if (= req.status_code 200)
  (for (kv (.iteritems req.headers))
    (print kv))
  (throw (Exception "Wrong status code")))

This code would ouput:

('date', 'Wed, 03 Apr 2013 12:09:23 GMT')
('connection', 'keep-alive')
('content-encoding', 'gzip')
('transfer-encoding', 'chunked')
('content-type', 'text/html; charset=utf-8')
('server', 'nginx/1.2.6')

As you can see, it's really simple to write Lispy code that really uses Python idioms.

There's obviously still a lots of missing features in Hy. The language if far from complete and many parts are moving, but it's really promising, and Paul's doing a great job implementing every idea.

I actually started to hack a bit on Hy, and will try to continue to do so, since I'm really eager to learn a bit more about both Lisp and Python internals in the process. I've already send a few patches on small bugs I've encountered, and proposed a few ideas. It's really exciting to be able to influence early a language design that I'll love to use! Being a recent fan of Common Lisp, I tend to grab the good stuff from it to add them into Hy.

OpenStack France meetup #2

Tue, 06 Nov 2012 00:00:00 GMT

I was at the OpenStack France meetup 2 yesterday evening.

This has been a wonderful evening, talking about OpenStack and all with around 30-40 people. I and Nick Barcet presented Ceilometer and have received some good feedbacks about it.

We should also thanks Nebula, who sponsored the evening, and Erwan Gallen since it was nicely organized, and free beers are always enjoyable.

Ceilometer, the OpenStack metering project

Fri, 27 Jul 2012 00:00:00 GMT

For the last months, I've been working on a metering project for OpenStack, so it's time to talk a bit about it.

OpenStack is a growing cloud platform providing IaaS. A problem easily identified by everyone building a public cloud platform is that nothing is provided to retrieve the platform usage data. Some data are available in some places, but not everything is, and you have to do a lot of processing from the various components to get something useful in the end. But in order to bill customers that are using your public cloud platform, you need to do his.

In this regard, a lot of companies running public OpenStack based infrastructure wrote their own solution to cover this functional areas, and to become able to bill theirs customers.

To avoid everybody doing and maintaining such a stack in their corners, the Ceilometer has been created.

The project aims to cover the metering aspect of the OpenStack components, pulling usage data from every components and storing them into a single place. It then offer a retrieving point for this data via a REST API.

The initial specifications have been written in April this year, and actual implementation started in May. The project is currently worked on by me, Dreamhost and Canonical.

We already have designed an architecture that we are implementing, and we hope to release a first usable version with Folsom.

I did a presentation of this project yesterday at XLCloud, which has been very well received.

If you are interested in helping us and contributing, feel free to join us during one of our weekly IRC meeting or fix some bugs. :-)

Lua workshop at Fabelier/tmplab

Thu, 14 Apr 2011 00:00:00 GMT

It seems I'll be at the Lua workshop at Fabelier/tmplab on April 28th 2011, where I'll try to present and talk about Lua, how to use it, and why you should probably not use it. ;-)

talks — jd:/dev/blog

Discovering the Tech Community in Toulouse: A Three-Year Journey

Sponsoring Conferences

Attending Conferences

Goals

COVID

Attending Conferences

Being in Charge

Attending FOSDEM 2020

Podcast.__init__: Gnocchi, a Time Series Database for your Metrics

Attending PyCon FR 2017

OpenStack Summit Boston 2017 recap

FOSDEM 2017, recap

Attending OpenStack Summit Ocata

How To Work Upstream with OpenStack

Python 3 in telemetry projects

Ceilometer deprecation

Aodh alarm timeout

Gnocchi improvement, past and future

PTG and beyond

Gnocchi talk at the Paris Monitoring Meetup #6

FOSDEM 2016, recap

Gnocchi talk at OpenStack Paris Meetup #16

OpenStack Design Summit Icehouse, from a Ceilometer point of view

Design sessions

Icehouse roadmap

Events management

Agents and group management

Documentation

Tempest testing

API improvements

Roll-up, archiving of data

Alarming

OpenStack Design Summit Havana, from a Ceilometer point of view

Design sessions

Havana roadmap

API extension

Healthnmon

Alarming

Hy, Lisp in Python

But what's Hy?

Does it work?

OpenStack France meetup #2

Ceilometer, the OpenStack metering project

Lua workshop at Fabelier/tmplab

Podcast.init: Gnocchi, a Time Series Database for your Metrics