Magnificent combo: Implementing Elasticsearch with Drupal Shankar Tue, 01/22/2019 - 10:58

Elasticsearch has been meritorious for The Guardian, one of the most reputed news media, by giving them the freedom to build a stupendous analytics system in-house rather than depending on a generic, off-the-shelf analytics solution. Their traditional analytics package was horrendous and was extremely sluggish consuming an enormous amount of time. The Elasticsearch-powered solution has turned out to be an enterprise-wide analytics tool and helped them understand how their content is being consumed.

Why is such a large organisation like The Guardian choosing Elasticsearch for its business workflow? Elasticsearch is all about full-text search, structured search, analytics, intricacies of confronting with human language, geolocation and relationships. Drupal, one of the leading content management frameworks, is a magnificent solution for empowering digital innovation and can help in implementing elastic search. Before we look at Drupal’s capability in implementing elastic search ecosystem, let’s unwrap Elasticsearch first.

Unlocking Elasticsearch

Elasticsearch is an open source, broadly distributable, RESTful search and analytics engine which is built on Apache Lucene. It can be accessed through an extensive and elaborate API. It enables incredibly fast searches for supporting your data discovery applications. It is used for log analytics, full-text search, security intelligence, business analytics and operational applications.

Elasticsearch is a distributed, scalable, real-time search and analytics engine - Elastic.io

It enables you to store, search and assess the voluminous amount of data swiftly and in near real-time. In general, it is leveraged as the underlying engine/technology for powering applications that have sophisticated search features and requirements.

How does Elasticsearch work? With the help of API or ingestion tools like Logstash, data is sent to Elasticsearch in the form of JSON documents. The original document is automatically stored by Elasticsearch and a searchable reference is added to the document in the cluster’s index. Elasticsearch API can, then, be utilised for searching and retrieving the document. Kibana, an open-source visualisation tool, can also be leveraged with Elasticsearch for visualising the data and create interactive dashboards.

Elasticsearch is often debated alongside Apache Solr. There is even a dedicated website called solr-vs-elasticsearch that compares both of them on various metrics. Both the solutions accompany itself with support for integration with open source content management system like Drupal. It depends upon your organisation’s needs to select from either one of them.

For instance, if your team comprises a superabundance of Java programmers, or you already are using ZooKeeper and Java in your stack, you can opt for Apache Solr. On the contrary, if your team includes PHP/Ruby/Python/full stack programmers or you already are using Kibana/ELK stack (Elasticsearch, Logstash, Kibana) for handling logs, you can choose Elasticsearch.

Merits of Elasticsearch

Following are some of the benefits of Elasticsearch:

• Speed: Elasticsearch helps in leveraging and accessing all the data at a fast clip. Also, it makes it simple to rapidly build applications for multiple use cases.
• Performance: Being highly distributable, it allows the processing of a colossal amount of data in parallel and swiftly finds the best matches for your search queries.
• Scalability: Elasticsearch offers provision for easily operating at any scale without comprising on power and performance. It allows you to move from prototype to production boundlessly. It scales horizontally for governing multiple events per second while simultaneously handling the distribution of indices and queries across the cluster for efficacious operations. 
• Integration: It comes integrated with visualisation tool Kibana. It also offers integration with Beats and Logstash for streamlining the process of transforming source data and loading it into Elasticsearch cluster.
• Safety: It detects failures for keeping the cluster and the data safe and available. With cross-cluster replication, a secondary cluster can be leveraged as a hot backup.
• Real-time operations: Elasticsearch operations like reading or writing data is usually performed in less than a second.
• Flexibility: It can pliably handle application search, security analytics, metrics, logging among others.
• Simple application development: It offers support for numerous programming languages comprising of Java, Python, PHP, JavaScript, Node.js, Ruby among others.

Elasticsearch with Drupal

For designing a full Elasticsearch ecosystem in Drupal, Elasticsearch Connector, which is a set of modules, can be utilised. It leverages the official Elasticsearch PHP library and was built with the objective of handling large sets of data at scale. It is worth noting that this module is not covered by security advisory policy.

Elasticsearch Connector module can be utilised with a Drupal 8 installation and configured so that Elasticsearch receives the content changes

Elasticsearch Connector module can be utilised with a Drupal 8 installation and configured so that Elasticsearch receives the content changes. At first, you need to download a stable release of Elasticsearch and start it. You can, then, move ahead and set up Search API. This is followed by the process of connecting Drupal to Elasticsearch with the help of Elasticsearch Connector module which involves the creation of cluster or the collection of node servers where all the data will get stored or indexed.

This is succeeded by the configuration of Search API. It offers an abstraction layer to allow Drupal to push content alterations to different servers such as Elasticsearch, Apache Solr, or any other provider that has a Search API compatible module. The indexes are created in each of those servers with the help of Search API. These indexes are like buckets where the data can be pushed and can be searched in different ways. Subsequently, indexing of content and processing of data is done.

Case Study

The website of Produce Market Guide (PMG), a resource for produce commodity information, fresh trends and data analysis, was rebuilt by OpenSense Labs. Interpolation of a JavaScript framework into the Drupal front end using progressively decoupled Drupal helps in creating a balance between the workflows of developers and content editors.

We rebuilt the website of PMG using progressively decoupled Drupal, React and Elasticsearch Connector module among others. 

To do the mapping and indexing on Elastic Server, ElasticSearch Connector and Search API modules were leveraged. The development of Elastic backend architecture was followed by the building process of the faceted search application with React and the incorporation of the app in Drupal as block or template page.

The project structure for the search was designed and developed in the sandbox with modern tools like Babel and Webpack and third-party libraries like Searchkit. Searchkit is a suite of React components that interact directly with your ElasticSearch cluster where every component is built using React and can be customised as per your needs. Searchkit was of immense help in this project. 

Logstash and Kibana, which are based on Elasticsearch, were integrated on the Elastic Server. This helped in collected, parsing, storing and visualising the data. The app in the Sandbox was built for the production and all the CSS/JS was integrated inside the Drupal as a block thereby making it a progressively decoupled feature.

Following the principles of Agile and Scrum, it resulted in a user-friendly website for PMG with a search application and loaded the search results faster.

Conclusion

The world is floating over a cornucopia of data. There is simply no end to the growth in the amount of data that is flowing through and produced by our systems. Existing technology has laid emphasis on how to store and structure warehouses replete with data.

But when it comes to making decisions in real time informed by that data, you need something like an Elasticsearch for searching and analysing data in real-time. Drupal can be a wonderful solution for implementing Elasticsearch ecosystem with its suite of modules.

We have been steadfast in our goals of empowering digital innovation with our suite of services.

Contact us at hello@opensenselabs.com to reap the rewards of Elasticsearch and ingrain your digital presence with advanced search capabilities.

blog banner blog image Elasticsearch Drupal Drupal 8 ReactJS Apache Solr Search API Elasticsearch Connector Blog Type Articles Is it a good read ? On

Join Agaric for Show and Tell Wednesday January 23rd from 11am - 11:30am EST (4pm GMT) Agaric friend and co-worker Rahul Baisane - Will present on building mobile apps using Drupal 8 He makes it look easy to build your mobile app with Drupal 8 - come to watch, or to share your own favorite projects, tips and tricks with us.   Meeting link is here: https://agaric.com/show See you there!!! The Agaric TeamDrupal version: Drupal 8.x

As promised 3 months ago, Gabe, Mateu and I shipped support for revisions and file uploads today!

What happened since last month? In a nutshell:

• 2.0 & 2.1 released :)
• Usage continues to rise: ~400 → ~1700 sites, 50% of those on 2.0 1
• Gabe proposed JSON:API profiles for versioning/revisions and multiple-arity relationships, to take advantage of the upcoming 1.1 version of the spec
• New core patch to bring JSON:API to Drupal core: #2843147-101
• Several refactors of internals 2 that pave the path for hypermedia links and partial caching
• JSON:API Extras is kept in sync — about 300 of you use that with JSON:API 2.x.

JSON:API 2.1

JSON:API 2.1 follows two weeks after 2.0.

Work-arounds for two very common use cases are no longer necessary: decoupled UIs that are capable of previews and image uploads3.

• File uploads work similarly to Drupal core’s file uploads in the REST module, with the exception that a simpler developer experience is available when uploading files to an entity that already exists.
• Revision support is for now limited to retrieving the working copy of an entity using ?resourceVersion=rel:working-copy. This enables the use case we hear about the most: previewing draft Nodes. 4 Browsing all revisions is not yet possible due to missing infrastructure in Drupal core. With this, JSON:API leaps ahead of core’s REST API.

Please share your experience with using the JSON:API module!

1. Note that usage statistics on drupal.org are an underestimation! Any site can opt out from reporting back, and composer-based installs don’t report back by default. ↩︎

2. Which we can do thanks to the tightly managed API surface of the JSON:API module. ↩︎

3. These were in fact the two feature requests with the highest number of followers. ↩︎

4. Unfortunately only Node and Media entities are supported, since other entity types don’t have standardized revision access control. ↩︎

• Drupal
• API
• Acquia

Tom Sliker started Broadstreet Consulting more than a decade ago, and has made Drupal a family affair. We dragged Tom out of the South Carolina swamps and into DrupalCamp Atlanta to get the scoop. How does Tom service more than 30 clients on a monthly basis with just a staff of five people? His turn-key Aegir platform, that's how! Join the conversation at https://www.lullabot.com/podcasts/drupal-voices/278-tom-sliker

Heading towards change - Upgrading or Upcycling? Vasundhra Mon, 01/21/2019 - 08:55

You’ve been living in a house for a while now and you are still fond of that place. But its no longer exactly what you need, and you are dreading for a change. 

You think of rearranging and modifying a few things. Maybe paint the room walls in vibrant colors or set up brand new furniture. And voila, it feels like home once again. 

A similar situation can be analyzed with the websites also. 

Websites are redesigned to keep up with the online trends. Sometimes it is done merely because the owners learn over time that the site design is not ideal for their purpose.

Coming back to the redesigning, people face scenarios where their house gets really old and is on the verge of collapsing. These type of situations demands them to move on to the next one exactly like the way a website owner does when he/she witnesses an end of its life cycle.

So what are these things? Is there a term for it? Is it a process? and what are the methods involved in it? 

Let’s see what exactly the terms are.

Introducing Upcycling and Upgrading 

It is true that a full rebuilt of a website is time-consuming and an expensive task just like a full rebuilt of the house is. In such situations you are only left with two options - either modify it to necessary changes or hop onto a completely new site.

If you choose the option which simply involves a refreshing appearance of the site or modifications in some parts then Upcycling is the hero for you.

To reuse the existing materials and create a product of higher value or quality than the original object or materials is termed as upcycling. 

In technical terms, Upcycling is an incremental approach to relaunch an existing website that is done to keep up with online trends, or maybe because of technical debt, or the reason where you have a well-established website and you have no plans of reconstructing it as a whole.  

Another option is to bring a website to cutting edge technology that is done for optimal results and an increase in revenues and profits. In other words Upgrading. 

Comparing Upgrading and Upcycling    Upgrading Upcycling When to do it? Can be performed when you start witnessing the end of a life cycle. Can be performed at any stage of an existing project Why to do it? Because the website contains bugs and is prone to hackers. To give the website a new appearance and improve the user experience. Rate of Investment You can see your investment as quickly as possible  The ongoing maintenance and upkeep are where you start to see your investment.  Performance Performance measurement takes several months and can be achieved with the help of different metrics Can be measured with the help of conversion rate and other metrics.  How to do it? Add more content, create multiple landing pages, go mobile friendly Allow to innovate and bring newer design versions for the relaunch of the whole site. When to do it?

Upcycling

If you have low cash but have time and some creativity in your mind you can reclaim almost anything and repurpose it. In other words, any website can be repurposed or redesigned at any stage of an existing project with existing website infrastructure. 

For a well-established web system that has been operational for several years, and doesn’t want to spend time and money to do a full rebuild, upcycling is for you. This brings improvement in editorial experience, user experience, front-end etc. 

Upcycling is also adopted due to the reason of investments. Implementation of investments is quick and easy.
 

 
Upgrading 

Computers and other electronics cannot be upcycled after they have come to an end of their life cycle, they must be upgraded to keep up with the technological advancements. Similar is the case with the websites. Websites are upgraded when it lacks functionality and speed, search engine fails to detect it or when it fails to meet lasting impressions. Upgrading a website is like a do-over of your brand.

Upgrading a brand new websites kills an enormous amount of time and leaves an organization with immense cost.

The reason why it faces such issues is evident, it is started from scratch and therefore requires deep research and analytics.

Why to do it?

Upcycling

Websites owner have plenty of reasons to transform their website. It might be due to the reason where they want a fresh appearance of their website, or because of the technical debt, or simply because they want to improve their user experience. 

Upcycling is the answer for all.

Though upcycling might de-prioritize the motto of “Big Bang Launch”, it meets one primary goal which is to reduce time to market for big website improvement.

Upgrading

Is your website prone to hackers and bugs? Is it preventing website visitors to find their way to your website’s content? - then it is time for you to upgrade it. Upgrading your website includes four main factors: Design, Marketing, Usability and Time. 

Design: It is the appearance of your website that reflects the business message which you are trying to portray. The design represents the collection of things like mobile responsiveness, compatibility with the browser, image resolutions

Marketing: Once the website design is done and the content is established, it is the time to promote it. Marketing involves all the SEO updates, measurement of effectiveness and a call-to-action (A button or link that is placed on a website to drive prospective customers to become leads by completing action on your landing page)

Usability: It is the term which describes the ease with which a particular website or a project is used. Usability helps in decreasing bounce rates and contributes to the performance. 

Time: A fresh website is a perfect way to save the admin’s time. This directly affects the customer service also. 

Rate Of Investment (ROI)

Upcycling 

ROI or rate of investment in Upcycling depends on the two factors sequentially:

• The cost and
• The results of a website. 

Cost is termed as the price of the website, which varies widely depending upon the budget and funds of the organization. Generally, the price of the website relates to the total time taken to create it. Factors such as project management, design, programming, team, content, CMS etc determines the cost of a website. 

Whereas the result also contributes to being an important factor in the whole journey of ROI. The website ROI may literally be negative if the site doesn’t produce the desired results which in return gives a loss in the cost. Factors affecting ROI are the cost of creating and maintaining the website, traffic, conversion rate, website lifespan etc.

When both these things, cost, and the result going overboard (and done right) it can generate so much demand that you would witness a steep increase in the website conversions. The ongoing maintenance and upkeep are where you would start to see your investment. 

Upgrading

Once you have taken a leap and invested in the reconstruction of your organization’s website it’s time to begin the calculation concerning ROI. While it is difficult to predict the precise ROI for a brand new website, it would definitely result in a host of benefits.

Benefits that might include an uptick in search engine ranking and a steady generation in the interest of potential buyers and customers. Though there are no dry methods to measure the ROI, there are different ways to track the success and determine whether you are meeting your goals or not.

Tracking the web activities, Monitoring search engines and calculating cost are some of the unique ways to do it. 
 

Performance 

Upcycling

Website owners who walk down the path of redesigning a website would agree with the fact that there’s a single most important metric that determines the success of a website: i.e Conversion Rate 

A conversion rate is the percentage of visitors to a website that completes the desired goal for the total number of visitors.

A high conversion rate is an indication of successful marketing and web design. It means people want what you are offering and that they are able to understand it. The whole point of redesigning a website is to improve the performance of it. 

Apart from conversion rate, there are certain other metrics which when optimized well would lead to better conversions and performance of your site. Metrics like:

• Number of session per user
• Pages per session 
• Less bounce rate 
• Per clicks around your website 

Upgrading

Let's assume that your organization has a great unique idea, you also proved that the demand of this must have “new product” you are selling is going to rock the market. You launched a beautiful website, and you wait for the rise in the performance graph.

Crickets. No traffic.

No traffic means no money and now you are in a dilemma that how would you mend it? Thus to improve it 4 important factors are required.

Backlinks: Backlinks are the incoming hyperlinks from one web page to another website that transfer equity to a website. Backlinks are the most significant factor that determines a site’s search ranking. The link-building strategy should be started as early as possible because it can take months for Google to update a rank. 

Content is the king: Google wants to see that your website should be dynamic and active for it to rank. Having a blog section on your website would do the task. You have to publish content frequently to give your customer a reason to return back.

Keywords: New websites have a hard time competing for top keywords, thus it is best to start with long-tail keywords. Keywords would allow you to attract the targeted audience who are looking for exactly that thing which you might be selling. 

How to do it using Drupal?

Upcycling

The Auraria Library main website is a great example of how upcycling can be achieved with the help of Drupal. The website was powered by Drupal 6, and it was beginning to see its end life. They needed an infrastructural upgrade with the highest priority. In 2016 a project team was established to kick off the journey of revamping the website. 6 months later the website was redesigned with a cloud-based, mobile friendly and localized ready content management system - Drupal 8.

Drupal 8 was chosen due to the fact that the organization enjoyed its scalability, flexibility and rapid availability with impressive community-driven service. Therefore, when a decision was to be made to select the CMS platform, they chose Drupal as their own. 

Thus by keeping longevity in mind, a swift change from Drupal 6 to Drupal 8 ( skipping Drupal 7) was made. 

Due to urgent migration, the project team implemented 80/20 law to break down the development objective into multiple releases. By doing so it not only enabled the possibility of a short first development cycle and helped in launching the website within six months but also bought them with the time to wait for highly contributed D8 modules to mature. 

User experience, increase in ROI, responsive designs, brand engagement, infrastructure improvement, and on-time delivery were the key objectives of the whole project. 

Upgrading 

Upgrading a website using Drupal involves several steps. First, you need to define the source site (make sure core modules like Migrate, Migrate Drupal, Migrate Drupal UI are enabled), then you need to review the pre-upgrade analysis. The upgrade of Drupal 7 to Drupal 8 should be done so that Drupal 8 site is empty. There might be a chance that conflicting IDs are detected, a warning about the conflicting IDs will be shown.

This warning can be dealt with two ways. One is where you ignore it and lose your data and the other one is where you abort it and take an alternative approach.

Now is the time for the upgrade. Depending upon the size and type of content the upgrade accordingly takes the time. 

Why Choose Upcycling Over Upgrading?

Now that you have read all the points and came to this section, a slight idea would be twirling around your mind on what is best for your website.

To give you a clear vision, let’s put it in simpler and accurate words.

To upgrade is to break something down and create a new product from the base material. This might sound great for consumer goods and other relatable things, but not always great for a website. Upgrading is a good choice for organizations that have the time and the money to invest in the whole construction, and also if they are starting to witness its end. For the well-established websites: not a good choice. 

The goal of upcycling is to add value without degrading the product. It is the concept of creating something new out of something old with as little effort as possible. Upcycling is the right choice for all the Drupal-based websites as it uses the features of Drupal 8 in a way that increases the conversions of the website. 

Upcycling and Decoupling 

If a website has a couple of complex backends logic that the website owner doesn't wish to rebuild but is eager to relaunch the frontend, upcycling is the solution. Relaunching the frontend as a decoupled site with Drupal themes and then integrate it with the backend makes more sense. 

Whereas if the backend really needs a major push, but you want to keep the existing frontend without the need of reconstructing it, upcycling could work for that too, after the backend is decoupled. 

Decoupling your entire architecture will enable the website to upcycle individual parts and bring great value to all the end users. It would highly contribute to things like

Infrastructure: Gets the most out of the existing content and website composition. 

User Experience: Improves user experience, design, and front-end without the need of waiting for any launch.

Investment: You are able to witness your investment as soon as possible. 

In the Nutshell

At the end of the day, we want our website to flourish in terms of performance and make improvements to reach the desired bars. 

Upcycling is like a blessing for all the website owners. At Opensense Labs, we help our clients with the transition of their website. Our services not only provides with splendid modifications but also takes care of all your needs and demands. Ping us at hello@opensenselabs.com 

blog banner blog image Drupal Drupal 8 Upgrading Upcycling Rate of investment Performance Decoupled Drupal Website redesign Blog Type Articles Is it a good read ? On

For this month’s South Carolina Drupal User Group I gave a talk about creating Batch Services in Drupal 8. As a quick side note we are trying to include video conference access to all our meetings so please feel free to join us even if you cannot come in person.

Since Drupal 8 was first released I have been frustrated by the fact that Drupal 8 batch jobs were basically untouched from previous versions. There is nothing strictly wrong with that approach, but it has never felt right to me particularly when doing things in a batch job that I might also want to do in another context – that really should be a service and I should write those core jobs first. After several frustrating experiences trying to find a solution I like, I finally created a module that provides an abstract class that can be used to create a service that handles this problem just more elegantly. The project also includes an example module to provide a sample service.

Some of the text in the slides got cut off by the Zoom video window, so I uploaded them to SlideShare as well:

Quick Batch Overview

If you are new to Drupal batches there are lots of articles around that go into details of traditional implementations, so this will be a super quick overview.

To define a batch you generate an array in a particular format – typically as part of a form submit process – and pass that array to batch_set(). The array defines some basic messages, a list of operations, a function to call when the batch is finished, and optionally a few other details. The minimal array would be something like:

<?php // Setup final batch array. $batch = [ 'title' => 'Page title', 'init_message' => 'Openning message', 'operations' => [], 'finished' => '\some\class\namespace\and\name::finishedBatch', ];

The interesting part should be in that operations array, which is a list of tasks to be run, but getting all your functions setup and the batch array generated can often be its own project.

Each operation is a function that implements callback_batch_operation(), and the data to feed that function. The callbacks are just functions that have a final parameter that is an array reference typically called $context. The function can either perform all the needed work on the provided parameters, or perform part of that work and update the $context['sandbox']['finished'] value to be a number between 0 and 1. Once finished reaches 1 (or isn’t set at the end of the function) batch declares that task complete and moves on to the next one in the queue. Once all tasks are complete it calls the function provided as the finished value of the array that defined the batch.

The finish function implements callback_batch_finish() which means it accepts three parameters: $success, $results, and $operations: $success is true when all tasks completed without error; $results is an array of data you can feed into the $context array during processing; $operations is your operations list again.

Those functions are all expected to be static methods on classes or, more commonly, a function defined in a procedural code block imported from a separate file (which can be provided in the batch array).

My replacement batch service

It’s those blocks of procedural code and classes of nothing but static methods that bug me so much. Admittedly the batch system is convenient and works well enough to handle major tasks for lots of modules. But in Drupal 8 we have a whole suite of services and plugins that are designed to be run in specific contexts that batch does not provide by default. While we can access the Drupal service container and get the objects we need the batch code always feels clunky and out of place within a well structured module or project. What’s more I have often created batches that benefit from having the key tasks be functions of a service not just specific to the batch process.

So after several attempts to force batches and services to play nice together I finally created this module to force a marriage. Even though there are places which required a bit of compromise, but I think I have most of that contained in the abstract class so I don’t have to worry about it on a regular basis. That makes my final code with complex logic and processing far cleaner and easier to maintain.

The Batch Service Interface module provides an interface an an abstract class that implements parts of it: abstract class AbstractBatchService implements BatchServiceInterface. The developer extending that class only needs to define a service that handles generating a list of operations that call local methods of the service and the finish batch function (also as a local method). Nearly everything else is handled by the parent class.

The implementation I provided in the example submodule ends up four simple methods. Even in more complex jobs all the real work could be contained in a method that is isolated from the oddities of batch processing.

<?php namespace Drupal\batch_example; use Drupal\node\Entity\Node; use Drupal\batch_service_interface\AbstractBatchService; /** * Class ExampleBatchService logs the name of nodes with id provided on form. */ class ExampleBatchService extends AbstractBatchService { /** * Must be set in child classes to be the service name so the service can * bootstrap itself. * * @var string */ protected static $serviceName = 'batch_example.example_batch'; /** * Data from the form as needed. */ public function generateBatchJob($data) { $ops = []; for ($i = 0; $i < $data['message_count']; $i++ ) { $ops[] = [ 'logMessage' => ['MessageIndex' => $i + 1], ]; } return $this->prepBatchArray($this->t('Logging Messages'), $this->t('Starting Batch Processing'), $ops); } public function logMessage($data, &amp;$context) { $this->logger->info($this->getRandomMessage()); if (!isset($context['results']['message_count'])) { $context['results']['message_count'] = 0; } $context['results']['message_count']++; } public function doFinishBatch($success, $results, $operations) { drupal_set_message($this->t('Logged %count quotes', ['%count' => $results['message_count']])); } public function getRandomMessage() { $messages = [ // list of messages to select from ]; return $messages[array_rand($messages)]; } }

There is the oddity that you have to tell the service its own name so it can bootstrap itself. If there is a way around that I’d love to know it. But really one have one line of code that’s a bit strange, everything else is now fairly clear call and response.

One of the nice upsides to this solution is you could write tests for the service that look and feel just like any other services tests. The methods could all be called once, and you are not trying to run tests against a procedural code block or a class that is nothing but static methods.

I would love to hear ideas about ways I could make this solution stronger. So please drop me a comment or send me a patch.

Related core efforts

There is an effort to try to do similar things in core, but they look like they have some distance left to travel. Obviously once that work is complete it is likely to be better than what I have created, but in the meantime my service allows for a new level of abstraction without waiting for core’s updates to be complete.

One of OSTraining’s customers asked how to implement content voting on their site. 

In Drupal 7, the popular choice was the Fivestar module, but that's still not ready for Drupal 8.

In this tutorial, I'll show you how to use the Votingapi Widgets module in Drupal 8. This module makes use of a “Rating” field, which you can customize and insert into your content.

There are multiple JavaScript and CSS libraries on the internet. They allow you to animate parts of your site and make them look more attractive.

The Animations module makes it easy to include animations on your Drupal site. 

There's a full list of the animations on the module page, but they include such wonderfully named animations as swing, tada, wobble, jello, bounceIn, bounceOut, fadeIn, rollOut, zoomIn, slideInRight and more.

In this tutorial, I'll show how to install and use the required libraries and the Animations module..

One of OSTraining’s members asked how to create a following web form with the Webform module:

• The site has four regions to be contacted: NE, SE, NW, and SW.
• Each region will have their own contact person/team.
• The user must be able to select one or more of the regions to contact at the same time, using checkboxes.
• All the submittions must be BCC-ed to the main administrator.
• If needed, the site owner should be able to add one or more email recipients for each region.

In this tutorial, you will learn how to meet these user requirements using the Webform module.

How do I get started with Drupal development?

That's a common question we get from people who join OSTraining for the first time. They want to know about the skills they will need, and what kind of classes they should take.

In this guide, I'll give you an overview to help you get started with Drupal development.

This article was originally written on 2017-04-12 but has been updated with current information and SEO best practices.

Drupal is phenomenal for SEO. When you use Drupal 8 for your content management system, you have a powerful tool to rock search engine optimization. I’ve worked in Drupal for 12 years and I’ve experienced firsthand how quickly search engines respond to a well-optimized Drupal website. I’ve seen customers triple their traffic in weeks after upgrading from another platform. I’ve seen competitive advantages from site-wide optimizations like RDF or AMP that put my clients on the cutting edge of SEO because they use Drupal. The benefits are a faster website, higher rankings and more traffic.

One of the main reasons Drupal is the content management system of... Read the full article: 6 Tips to Rock Drupal 8 SEO

Request for Proposal: Skill development for disadvantaged youth | HSBC Skills for Life: 2019

Dear Changemaker,

The Swades Foundation in partnership with HSBC is inviting proposals from NGOs, for skills projects that have specific emphasis on empowering youth from disadvantaged communities/ background through skills training.

We welcome you to explore this opportunity for financial support, please visit www.swadesfoundation.org/hsbcskillsforlife for detailed information on the programme, eligibility criteria and the process to apply for grant. The deadline to submit your application is January 21, 2019.

Click here to download the application pack.

You may also circulate this RFP with other NGOs who you feel might be eligible to apply.

More about HSBC Skills for Life in partnership with the Swades Foundation

HSBC Skills for Life is a five year INR 100 crore commitment by HSBC India that will focus on employment linked skill development of disadvantaged youth, upskilling of teachers and educators, and women’s empowerment through financial education and livelihood enhancement. Under the umbrella of this programme, HSBC and Swades have forged a five-year INR 50 crore partnership that will skill many thousand disadvantaged youth across the country and indirectly impact and inspire many others.

As part of this joint programme, HSBC and Swades have announced this ‘Request for Proposal’ seeking proposals that will focus on employment linked skill development of disadvantaged youth. The grant available ranges from INR 25 lakh to INR 75 lakh, for projects with duration of 12 to 18 months.

Across the previous three grant cycles (i.e. 2016, 2017 & 2018), we have approved grant of over INR 33 crore to 68 NGOs across 21 states.

NGOs registered in India as a Trust, Society or a Section 25/ Section 8 Company, and having a valid FCRA registration only are eligible to apply.

Read more about this grant opportunity at www.swadesfoundation.org/hsbcskillsforlife

With best regards,

Manager – HSBC Skills for Life

Phone : 022 61093731 [Mon – Fri; 10 AM – 5 PM]

Email : skills@swadesfoundation.org

---------------------------------------------------------------------------

‘HSBC Skills for Life’ in partnership with the Swades Foundation aims to provide employment linked skills training to disadvantaged youth.

To kick off 2019 properly, the Experience Express is taking a break from Drupal and web development to consider an oft-forgotten component of new digital experiences in the conversational space. Though many organizations, some of Acquia's customers included, have leapt headlong into building conversational interfaces, sometimes it can be difficult in such a newfangled paradigm to consider all possible angles where things can go awry.

Tags: acquia drupal planet

Drupal South 2019 / Round Two Australian Drupal community and market are growing stronger. Participated in Drupal South Event for a second time, where I learned more about GovCMS and presented about Gatsby to create a React Application to accelerate integrations. enzo Fri, 01/18/2019 - 12:16

One of our customers asked how to build a mega menu in Drupal 8.

Mega menus are menus with multi-column navigation. They are fastly becoming a trend in web design.

The Ultimenu module allows you to insert Drupal blocks into a menu. You can easily use it to build fairly complex menu layouts.

In this tutorial, you will learn how to build a simple mega menu using the Ultimenu module and Bootstrap.

Drupal 7 core, together with Drupal contrib, has the ability to ship a powerful platform for our digital requirements in the enterprise world. The multilingual feature is one of the prime reasons that Drupal is a distinguished and a preferred solution to achieve business goals. Drupal 8 includes the multilingual package in the core itself, but Drupal 7 requires some extra amount of effort to build this package as a whole to make Drupal multilingual ready.

Tags: acquia drupal planetmultilingualdrupal 7custom entity

This theme blows your mind. It is the source of all creation, all that a themer holds important: semantic HTML. It is the Big Boom.

When we say DrupalCon, the upcoming DrupalCon Seattle 2019 event is probably what first comes to mind. But while we have been selecting sessions, setting up BoFs, and letting you know about the additions to our Con, we at the Drupal Association have also been looking ahead to DrupalCons of the future. We are excited to share those with you now.

In the past, we used to announce the next DrupalCon location during the closing session of the previous Con. This was a lot of fun, but created some logistical problems for the events team, and made it difficult to do all the work we need to do to secure our next con locations. It is a multi-year process to secure a venue for DrupalCon, so we've made some changes that help us coordinate with venues, hotels, and partners without relying on a veil of secrecy.

You first saw this change during the DrupalCon Nashville Closing Session, where we announced both 2019 (Seattle) and 2020 (Minneapolis).

We're taking these changes a step further by looking far into the future to announce the North American DrupalCons for 2021, 2022, 2023 and 2024. We're thrilled to announce the selected cities, as well as share the process that went into making these selections.

Where DrupalCon is going

Together with each of our partner cities, we're excited to announce the upcoming locations for DrupalCon North America:

• DrupalCon Boston 2021 (April 16-21)
• DrupalCon Portland 2022 (April 25-29)
• DrupalCon Pittsburgh 2023 (June 2-9)
• DrupalCon Portland 2024 (May 6-10)

Want to understand the process that goes into city selection? The search for each location starts four or more years before the event, and you can read on for the inside scoop into how this plan came together. Wondering why all the selected cities are in the USA? We encourage you to read our prior blog about why the sustainable choice for North American locations is in the United States for the foreseeable future.

How we got here  Planning for the future

Historically, DrupalCon locations have been contracted a couple of years before they happened, in a city where we were excited to host the community, that we revealed in a fun fashion at our Closing Session the prior year.

However, announcing the new event only a year in advance—and selecting new cities for every event—created some logistical problems; conference center venues large enough to host DrupalCon are often booked four or more years in advance. This has meant that cities we would love to visit have often simply been booked during the dates that would work with our community needs, or are too expensive because we couldn't make multi-year commitments - which took a lot of options off the table.

In benchmarking ourselves with comparable conferences (in size, audience, and program), it became clear that many established organizations were booking multiple years in advance. This is in part due to the availability of desirable cities, but also that securing locations in the future equates to more competitive pricing.

As the Drupal Association matures and starts leading change in the community and in the open source world, we determined it was best to look farther into the future for our largest annual conference.

Creating a location pattern that the community can count on

We took a serious look at data from past attendance, the locations we're trying to reach, and where we see the most traction from the community.

In analyzing data from DrupalCons dating back to DrupalCon Austin 2014, we were able to deduce some high-level insights about our attendees:

• 88% of attendees at DrupalCon North America come from the United States
• In hosting a DrupalCon in a coastal city in the USA, attendance from the regional community local to those cities can be 13-17% higher than the regional attendance we see in other cities (not counting those who travel greater distances).
• Conversely, when hosting a Con in the center of the country, attendance decreases significantly from the coastal audiences and does not significantly increase from the hosted-area region.

With the majority of our conference attendees in mind, we set out to host our conference in coastal cities, where, by ‘showing up’ our community has proven they want to go. This led us to primarily work on sourcing East Coast and West Coast cities for the upcoming years.

With our upcoming DrupalCon Seattle 2019 on the West Coast, and DrupalCon Minneapolis 2020 in the middle of the USA, we aimed to host DrupalCon 2021 on the East Coast, and from there, jump between the coasts for the foreseeable future.

The benefits of repeating cities

As we did with timing and location, we also stepped back to ask ourselves, why do we move this conference every year? The logical answer is that it makes the conference more accessible to new audiences in different areas. But our past attendee data doesn't support this conclusion. So we asked ourselves again: If it isn't bringing in large numbers of new first-time attendees, why do we search for a new city every year?

We had heard anecdotally that it was because ‘Drupalers like to go on vacation in new cities’ and that ‘it helps grow the community in a new city,’ but these answers aren't well supported by the data, so we decided to re-evaluate our strategy.

When we release our RFP to the world, we work internally with the Drupal Association Board to determine our Selection Criteria. A lot of this hasn’t changed because the Con hasn’t changed drastically in a few years. The top 5 things that we evaluated in each city’s bid were:

• Large/versatile venue - Could the venue fit our 150+ sessions, 3,000 people for lunch, and the breadth of programming offered at our Cons?
• Popular tourist area - Do people want to go there? Is there a wealth of activities for them post-sessions each day?
• Strong business community - Do we already have partners in the city? Is it a place our sponsors have expressed as a city where they’d like to do business?
• Tech-focused city - Is the city supportive of tech and open source? Are there businesses and organizations that may participate in our event because we are in their city?
• Large and strong Drupal community - Does this city have a community that has hosted a successful camp in the past? Is there a solid community that regularly meets and would help support the planning of a Con?

It had been a few years since we selected new DrupalCon cities, so reviewing and updating the criteria seemed prudent. We added and changed the following criteria:

• CHANGE: In the venue criteria, we included the ability to change the program around, since as the community grows and changes, we want to be able to flex our program.
• ADD: Welcoming to all attendees. We wanted to make sure that topics such as legislative actions, political climates, and inclusiveness of the cities were taken into account to ensure that we were placing our Con in a city where all members of our community would feel welcome.
• CHANGE: When DrupalCons were mostly managed by the community, the need for a large and strong community was imperative to the success of a Con. Since the Drupal Association has taken on the bulk of planning, pricing, and executing of the Con, the need for the community to be of a certain size is no longer a qualifying factor. In fact, by changing the focus, we could look at cities that didn’t have large community groups at the time, but maybe a Con could inspire one.

When we examined our search criteria and started matching it up with real cities that we could reach out to, the list became short. With that reality, it became apparent that we would need to begin repeating cities. We seized the opportunity to proactively address that reality.

In speaking with tech event leaders from other communities and organizations, it was helpful to get a fuller understanding regarding the benefits of repeating a city location:

• Time: Securing multiple years can save an organization time, money and peace of mind. By doing this, you eliminate the need to do site visits and RFP gathering again the following year.
• Staff Capacity: By hosting an event somewhere you’ve already been, the staff does not need to learn a new floor plan, crew, process, regulations, etc. It is estimated that in eliminating these normal challenges of a new venue, that the staff capacity can be reduced by 25%, freeing them up to focus on the event itself.
• Negotiation: Planners can gather information on the facility once and focus on strategic negotiating, which translates to consistent concessions and commissions with minimal increases in rates/pricing annually.
• Cost Savings: Event budgets can be determined early, giving the planner more time to focus on the important things like planning for the success of the event. And, if you have done all of this well in the beginning, you will have the peace of mind to know that you are prepared for surprises that inevitably come along.
• Relationships: Multiyear contracts require a partnership. Planners, venues, and hotel partners can create a strategic plan to build the event and their services. In working with a crew for more than one year, improvements can be made and the crew is better prepared to serve the attendees the next time around.

Getting from ideas to contracts

We released our RFP on August 13, 2018, on the Drupal Association blog. It was also sent to multiple cities that met our criteria. Within our RFP, we shared our tight timeline, with the goal of signing contracts for 2021, 2022, 2023, and 2024 by the end of 2018. Below is a glimpse into the work that transpired between launch date and sign date.

• September 4, 2018. Is the date that we requested cities submit their detailed bids. Per our RFP, we had multiple questions about space needs, catering, AV, diversity and inclusion, internet, hotels, and more.

• September 5-11. Partnering with our fantastic production partner, Groundswell Marketing, we reviewed the proposals to see if any questions arose initially about the information provided. Most proposals were 30+ pages of information with pricing grids, proposed hotel blocks, and ‘why our city’ info. We ask each city for some hard numbers on regular items like a gallon of coffee or the hourly rate of an AV technician. This helps us immediately get a picture of a Con cost.

• September 11 - September 17. For cities we hadn’t been to before, the next step was to interview the city via a Zoom call to better understand how they were a good fit for our conference. Questions like ‘how would attendees be made to feel welcome in your city?’ and ‘How easy and affordable is it to get from the airport to the Convention Center?’ are asked in our initial determination.

• September 18 - 21. Once we determined cities that met our criteria, we dug a bit deeper into each city’s numbers. We laid out the entire Con on their floor plans to determine if we could fit and how. We inquired about real quotes for line items like our AV and our internet. We costed out catering estimates and space rental.

• September 24 - October 26. With our list in hand, we did our due diligence visiting possible future Con locations. In these meetings we reviewed the space and discussed how the attendee experience would feel. We met representatives from various departments of the Convention Center’s team to negotiate pricing and discuss pain points. We also did a whirlwind tour of the city to get a feel of what attendees would see/do after the Con each day.

• October 29 - November 14. We reviewed all of this information with the Drupal Association leadership team and collaboratively determined the priority of cities based on our search criteria. Going further, we then included data points on pricing, incentives, conference dates, etc, to come to a final recommendation for each year.

• November 15 - December 3. Built a working budget with concrete numbers for each preferred city to get an accurate future picture of finances. This involved getting future pricing on catering, network, hotel rates, etc. Worked back and forth with the city to negotiate down pricing.

• December 5. Presented recommendation to the Drupal Association Board for buy-in and support.

• December 6 - 27. Requested contracts from our preferred venues and hotels. Each city has one Convention Center, and at least 5 hotels, so in asking for these hotels, we had about 20 contracts to review. By looping in our legal team and our insurance group, we were able to further negotiate terms that make committing to future years smart, sustainable, and safe.

• December 28. Signed the last contract and sent it off to the cities. Signing before the end of the year met a contracting deadline that gave us a lot of financial benefits.

• End of December. CELEBRATED the end of this intense and action-packed process, and the future of sustainable and secured DrupalCon programming.

Serving our community

We aim toward growing adoption, one of the Drupal Association’s main goals. In planning ahead and setting ourselves up for a sustainable and fiscally responsible future conference plan, we can allocate our resources better to focus on creating a successful event that drives to this goal. By making these decisions now, we work to strengthen the foundation of the Association in order to continually work to serve our incredible and growing open source community.

We appreciate the questions and interest that community members have had in this process and were happy to do a deep dive to show you the planning, strategy, and work involved in selecting a city for a future DrupalCon. We invite you to share your thoughts and comments below, and we look forward to seeing you at a Con in the future.

This tutorial is all about managing users on your Drupal 8 site.

I'll show you how to control who can do what on your site:

• Who can create, delete, and edit content?
• Who can upload modules and themes?
• Who can modify menus and blocks?

You also see how to make user accounts more interesting. You do this by allowing users to add more information about them. 

If your library hosts community programs and events, you should check out Intercept: a new product for helping libraries run better events. Intercept makes it easy to create and manage events. Even better, it provides actionable reports to help measure success and recommend strategic improvements for your programs in the future.

Intercept Features

Intercept provides valuable features for event management, equipment reservations, room reservations and customer tracking.

Event Management

• Easily add and edit events.
• Book rooms at the time of event creation.
• Host events at outside venues.
• Make custom templates for quickly creating future events.
• Create recurring events.
• Set up registration for events, including waitlists.
• Browse events as a list or grid-style calendar.
• Filter events by type, audience, location, date and keyword.
• Save and register for events.
• See similar events based on type and location.
• Receive recommendations for other events based on your preferences, events you’ve attended and/or saved.
• Analysis for events.

Equipment Reservations

• Browse and reserve available equipment.
• Set reservation periods by item.
• Manage and approve requests.
• Report on equipment usage.

Room Reservations

• Customize rooms and locations.
• Browse rooms by type, capacity and timeframe.
• Reserve rooms with validation to ensure rooms are not double-booked.
• Deny or approve reservation requests, with email notifications.
• Staff reservations are automatically approved.

Customer Tracking

• Ability to integrate with popular Integrated Library Systems (ILS).
• Integrates with Polaris ILS.
• Single sign-on with website and ILS.
• Allow attendees to scan into events with their library cards.
• Gather and analyze feedback from customers.
• Analyze event attendance numbers with population segmentation
• Download a CSV report on attendance.

Built By Libraries, For Libraries

Intercept was built as one part of a large redesign and redevelopment project with Richland Library. From the beginning, Richland’s vision was to both create a product to help measure the effectiveness of its own events, and to release that product to the wider community of public libraries. More than five years of user research, planning and beta testing have gone into this product to date. Intercept was designed and developed by a team intimately familiar with the problems that it solves.

Open Source

Intercept was architected as a suite of modules for Drupal 8, is open source, and is freely available to all. You can download an early version of the code from Drupal.org at https://drupal.org/project/intercept, and the most recent version will available there soon. If you’re interested in learning more about using Intercept to help make your library’s events even better, we’d love to help! Just drop us a line on our contact page and we’ll be in touch right away.