Configure Apache Solr with Drupal for better content search

There have been times when clients are not very satisfied with the default Drupal search as it does not meet their requirement especially when you need specific search control beyond core e.g. facets and related content. In order to resolve this issue and make search fast for end customers, we can use Apache Solr - an open source enterprise search platform - and configure it with our Drupal site.

In one of our previous blog post, we have discussed “How To Create Custom SOLR Search With Autocomplete In Drupal 7”. Note, before creating custom Solr search, we need to set-up Solr in the server and configure with Drupal, and this is what I am going to talk about on this blog. Here, we will also learn to create the core in Solr. Unlike Ubuntu, you can also set-up Solr on Windows machine.

You may also like to check out our previous post on Installing & configuring Apache Solr-5.2.0 with Drupal 7 using Search API on Ubuntu 14.04.

Note: In the above-mentioned blog, Apache Solr - 5.2.0 is configured with Drupal 7 and in this blog, I am configuring Apache Solr - 6.6.1 with Drupal 8.

Prerequisite

Setting up the Java environment

As we all know, Solr is a Java application and to setup Solr we need Java runtime environment.  Also, we need to install Python software properties to install the latest Java 8. 

You can run the following commands to install Python software.

sudo apt-get install python-software-properties

After executing the command, add the webupd8team Java PPA repository in your system by running:

sudo add-apt-repository ppa:webupd8team/java

Then update the package lists to fetch the available packages from the new PPA:

sudo apt-get update

Now install the latest version of Oracle Java 8 with this command:

sudo apt-get install oracle-java8-installer

After installing Java, check the version by running the following command:

java -version

Installing Solr application

Step 1 Download the latest version of an available package from Apache web page using the wget command. For this setup, I am using Apache Solr 6.6.1 

Cd /tmp
Wget http://www-us.apache.org/dist/lucene/solr/6.6.1/solr-6.6.1.tgz

Step 2 Now run the below-mentioned command to extract the service installation file:

tar xzf solr-6.6.0.tgz solr-6.6.0/bin/install_solr_service.sh --strip-components=2

Step 3 Install Solr as a service using the script:

sudo ./install_solr_service.sh solr-6.6.0.tgz 

Step 4 Use the given below command to check the status of the service

service solr status

Step 5 After installing the Solr, create the core in the Solr.

Note: This is an important step so that we can index the contents to the Solr

By running the following command, core will be created in the solr server. 

Here, first_solr_core is the name of the core

sudo su - solr -c "/opt/solr/bin/solr create -c first_solr_core -n data_driven_schema_configs"

Now that Solr core has been created, you can check it in the solr dashboard.

Apache solr dashboard

Configuring with Drupal 8

After installing Solr in Ubuntu you need to configure it with Drupal 8. Let’s see how to do it step-by-step:

Step 6 Download the search api solr module and copy the files from Solr-conf directory respective to Solr version installed. 

Example: In this, we will copy all the files from search_api_solr/solr-conf/6.x and put that files under /var/solr/data/first_solr_core 

Step 7 Restart the Solr server by using the following command:

Sudo solr service restart

Okay, so now you know how to setup Apache Solr in Ubuntu and configure it to speed up your Drupal search. Just to let you know, for me it worked and I successfully configured Solr with Drupal site for one of our client. Go and try Solr on your website and see what it can do for you. Do share your experiences by commenting on this post.
 

Continuous integration using Jenkins and GitHub to automate deployment

Continuous Integration (CI) is rapidly becoming an integral part of software development process as it makes our monotonous and repetitive tasks a little less grindy. CI is a project development practice where developers integrate code into a shared repository frequently. Each Integration is then verified by an automated build that allows the team to detect problems in an early stage. 

This post will walk you through the continuous integration with Jenkins and GitHub. Here we will install Jenkins, create Jenkins task and then configure it with GitHub.

Let’s get our arms around why Jenkins is so popular and why this is not just a hot topic, but an important best practice for developers.

Related: Git and GitHub for Beginners

Why Jenkins is so popular for Continuous Integration?

Jenkins is an open-source tool that tests and compiles the code. If everything is fine then it deploys the code for production else you will get a notification that builds has failed and you need to fix the bug/error. Similarly, Selenium - an automation testing tool - minimizes the testing time and eliminates repetitive human tasks. 

Jenkins has an array of advantages, such as:

  • Easily configurable
  • Platform independent
  • Rich plugin support
  • Easy to create new Jenkins plugin if one is not available

Now you know why Jenkins is so popular, go ahead and install it. 

Step-by-step guide to Install Jenkins

Follow the below steps:

Step 1: First install Java, using the following command. 

$ sudo apt-get install openjdk-8-jre-headless -y

Step 2: Add the key and source list to apt for Jenkins.

$ wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -

Step 3: Now create source list for Jenkins, using the below commands

$ sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'

Step 4: Update the packages

$ sudo apt-get update

Step 5: Now install Jenkins.

$ sudo apt-get install jenkins

Step 6: After installing Jenkins. Go to: http://your_ip_address:8080 and run the following commands to get admin password.

$ sudo nano /var/lib/jenkins/secrets/initialAdminPassword

Step 7: Now copy the password and paste into browser.

Getting started with jenkins

When finished, you should see a screen similar to the below one.

Jenkins home page

Integrating GitHub with Jenkins

The integration of GitHub with Jenkins automates deployment, testing and improves products quality while saving a significant amount of time of developers. Follow the below steps to integrate GitHub with Jenkins:

Prerequisite: Install GitHub Jenkins plugin. 

Step 1 Go to Manage Jenkins -> Manage Plugin.

Manage Plugin

Step 2 Search Github Plugin in the Available tab then click on Download now and install after the restart.

github plugin

Creating a Jenkins job

Step 1 To Create a new task for Jenkins, click on “New Item” then enter an item name that is suitable for your project and select Freestyle project. Now click Ok. 

Enter item name

Step 2 Select the GitHub project checkbox and set the Project URL to point to your GitHub Repository.

Github project creation

Step 3 Under Source Code Management tab, select Git and then set the Repository URL to point to your GitHub Repository.

GitHub project management

Step 4 Now Under Build Triggers tab, select the “Build when a change is pushed to GitHub” checkbox.

Build triggers

Step 5 At the end, execute Shell script to take a clone from dev. When the configuration is done, click on save button.

Build environment

Configuring GitHub for Jenkins

Step 1 Install the Jenkins (GitHub plugin) on a git repository.

Jenkins (GitHub Plugin)

Step 2 Now set the Jenkins hook URL as the URL for your machine. 

Note: Github will not accept Local URL, hence, we need to install ngrok software to create live URL for port 8080. Here you can find complete guide for configuration: https://ngrok.com/download

Add Jenkins

Phew. That was quite the ride!

Every time you publish changes to GitHub, it will trigger new Jenkins job. Now you know an entire process of continuous integration with Jenkins and GitHub. Similarly, you can integrate Jenkins with selenium to automate testing. Go ahead and try it on your own software development. Note that computers don't get bored so while they handle deployment and testing, you're free to do other important works.

Related: Drupal 8 applications using Drush Aliases

Is there anything you'd like to clarify or see further clarified about Jenkins and GitHub? Let me know in the comments below!

Below given is a presentation on Continuous Integration with Jenkins and GitHub

Selenium: A beginner guide to automation testing tool

Before delving into the how of automation testing using Selenium, let me talk about the why.

Over the past couple of years, the demand for automation has increased at an unprecedented speed and scale as it indispensably minimizes the testing time, eliminate repetitive human tasks and make life easier. The advent of an open source automation testing tools, such as Selenium, has significantly reduced the demand and scope of manual testing.

Needless to say, every testing has its own quirks and best practices! However, there are certain standard best practices that generally apply to most automation, too. Let’s review the best practices of automation testing. You may also like to check out our previous blog post on executing Automation Testing using Selenium.

What is Selenium and why we chose it over other automation testing tools?

Selenium is an extensively used automation tool because of a number of reasons besides being free. The open source tool is supported by a great community and allows QA engineers to write/run test cases in Java, C++, perl, PHP, Python, Ruby, and Scala.

The growing demand of automation industry has persuaded many market leaders to come up with better alternatives. However, all are not open source and free. Check out the list below for the paid automation tools: 

  • TestingWhiz by Cygnet Infotech
  • HP - QTP by HP
  • TestComplete by SmartBear Software
  • Ranorex by Ranorex GmbH

Test cases in Selenium

Implementing test cases in Selenium using Java is not that difficult, but one needs to know how to locate an element in the browser and take required actions.

Locating elements in browser using Selenium

Well, there are several ways we can locate elements in Selenium. Below-mentioned is the most common methods:

  • By ID:
    • Syntax : driver.findElement(By.id(“element id”))
  • By CLASS:
    • Syntax: driver.findElement(By.className("element class"))
  • By NAME:
    • Syntax: driver.findElement(By.name("element name"))
  • By TAGNAME:
    • Syntax: driver.findElement(By.tagName("element html tag name"))
  • By CSS Selector:
    • Syntax: driver.findElement(By.cssSelector("css selector"))
  • By Link:
    • Syntax: driver.findElement(By.link("link text"))
  • By Xpath:
    • Syntax: driver.findElement(By.xpath("xpath expression"))

Xpath

XPath is defined as XML path. In Selenium automation, Xpath is a syntax to find out any element on the web page using HTML DOM structure if the elements are not found by the common locator methods like id, class, name, etc. 

The basic format of XPath is explained below.

Xpath=//tagname[@Attrribute=’value’] 

To use Xpath in Firefox, we need to install FirePath. Currently, FirePath is not supported by Firefox 56 so avoid updating your browser.

Prerequisites:

  • Eclipse IDE
  • JDK 1.7 or more
  • Latest Selenium JAR (3.5.3)

Here I have created a small test case for the demo where the following functionalities are supposed to work.

The test case is to:

  1. Open “http://example.com” in web browser
  2. Enter first name and last name
  3. Click on submit

To open chrome browser through Selenium, you need to download chrome driver from “https://sites.google.com/a/chromium.org/chromedriver/downloads”.

Github link for the complete project: https://github.com/chirag-munjayasara/automation-testing

Creating project to write test case

Following are the major steps involved in creating a project in selenium:

  • Open IDE
  • Launch your own Workspace
  • Create a new JAVAEE project
  • Give project name
  • Click next
  • Click next
  • Click finish

Your project is created. Now you can create java file in the package.

Right-click on the project name -> new -> package. Give package a name and then click finish.

When you will expand your project folder you can see the below structure.

      Project Name
                   Java Resources
                              Src
                                         Package name

                               Libraries

Now create java file to write test cases.

Follow the below steps to create a java file:

  • Right click on package name -> new -> class
  • Give class name (xyz)
  • Click finish

After creating java file, you will see the below directory structure.

      Project Name
                   Java Resources
                              Src
                                         Package name

                                                      xyz.java

                              Libraries

Follow the below steps to add Selenium libraries to the project: 

  • Right click on Src folder
  • Build Path -> Configure Build Path
  • Click on Libraries Folder
  • Click on Add External JARs
  • Select the Selenium JARs that you have download
  • Click Apply and Close.

Selenium library structure

  Project Name
                   Java Resources
                              Src
                                         Package name

                                                      xyz.java

                              Libraries

                                         Referenced Libraries
                                                    Selenium-Server Standalone-3.5.0.jar

Your project is ready. Just write below test case and run in Selenium. :)

Example 1: Here I have used “By.name” to locate text box and “By.id” to locate submit in the browser.

By.name("firstname").sendKeys(‘localtechx’);
By.name("lastname")).sendKeys("Bound");

By.id("submit")).click();

See the screenshot below for an example:

SendKeys
SendKeys is  used to pass value in the text-box.


Use By.id method and perform click operation to locate submit button.


Example: 2 Selecting radio button, checkbox, select option, locate anchor tag and pause the execution.

Following is an example to locate HTML elements by using different methods like By.partialLinkText, By.tagName, By.xpath etc.

Lastly, to generate proper test results and understand the output, we need to use test framework - TestNG - as Selenium test does not generate proper format of test results.

Inspired from JUnit framework, TestNG covers all kind of tests like unit, functional, end-end, integration etc…

Advantages of TestNG over JUnit

  • TestNG annotations are easier to understand when compared to JUnits
  • Generate logs
  • Create an HTML report
  • Dependent method

Installing TestNG in Eclipse (why we need to install in Eclipse and what is the use of it?)

  • In the menu bar, click “Help”
  • Select Eclipse Marketplace option
  • Type TestNG in the dialog box and then press enter
  • Click install
  • New dialog box will pop up, do not change anything and click on confirm
  • Click finish by accepting the license.
  • After installing, Restart the Eclipse.

How to create TestNG project and check whether above-mentioned cases run perfectly or not.
              
Follow the below steps to create TestNG project:

  • Right-click on your package name, you will find the option TestNG
  • Hover it and click on Create TestNG class
  • Select necessary options
  • Click on finish.

Console output

Console output

HTML report

HTML Report

 

testng.xml

To handle multiple test classes, we need to create Testng xml file. After creating TestNG file, configure test run, set test dependency, include or exclude any test, method, class or package and set priority in the xml file.

Follow the below-mentioned steps to create an XML file and run test case:

  • RIght Click on your Java file, you will find the TestNG option.
  • Hover it and now select “Convert to TestNG”.
  • Your testng.xml has been created.
  • Now whenever you want to run your test case, just run this file. This will give you the same output as your .java file.

See the following screenshots for an example:

Convert to TestNGConvert to TestNG step 2

 

Basically, automation testing is an automation of a manual process. In software testing, in fact, it helps to execute pre-scripted tests on an application before its release into production. Though the scope of manual testing is decreasing, it is still prevalent in some of the industries. Testing method totally depends on the project requirement, budget associated with the project, and others.

This is what automation testing is all about.

Now you know the entire process related to automation testing using Selenium. I want to hear from you, which of the testing - automation or manual - have you find most useful? Please share in the comments section below.

Below given is a presentation on automation testing using Selenium.

References:

Extend existing field widgets in Drupal 8 application using annotation plugin

Have you ever wondered how the text or email or entity reference field is extended in Drupal 8? Or how to create a custom field/widget/formatter so that it can match with the rest of fields in your Drupal application? This blog will cover everything required to extend existing field widgets in Drupal 8 using annotation plugin. 

Many developers, who recently started working on Drupal 8, may not be aware of an entire process so let’s take a closer look to everything step-by-step. Key comparisons between Drupal 7 and Drupal 8, what is an annotation, why annotation and sample use case from an Inline Entity Form step-by-step. After completing this post, you will be able to extend the field with your own methods/functions without disturbing contributed module code.

Note: The below-mentioned functions are used to access properties of available widgets, which are moved to methods on the Widget Plugin Manager.
Comparisons

In Drupal 7

  • hook_field_widget_info() is replaced by annotation-based plugin 
  • hook_field_widget_settings_form(), hook_field_widget_form(), hook_field_widget_error() : replaced by Widget Interface methods

In Drupal 8

  • hook_Info is replaced by annotation-based-plugin
  • public function settings Form(array $form, array &$form_state)
  • public function formElement(FieldItemListInterface $items, $delta, array $element, array &$form, array &$form_state)
  • public function errorElement(array $element, ConstraintViolationInterface $violation, array $form, array &$form_state)

Let’s take a brief look at Annotations as it plays a significant role in extending FieldWidgets.

Annotations are one of the new concepts in Drupal 8, which are written as  PHP code comments above a class or function. Annotations contain metadata about the function or class.

Note: In Drupal, we have an option to add metadata about the functions inside the comment block so that it can automatically take annotations when Field Widgets rendered on the page. It will help in avoiding multiple files to declare the metadata information.

Why Annotations?

The annotation meta-data lives in the same file and is an integral part of the class that implements the plugin. 

This makes it easier to find and to create a new custom plugin by simply copying an existing one.

In addition, Implementation used by Drupal to parse the annotation simply tokenizes the text of the file without including it as a PHP file, so memory use is minimized.

The annotations syntax comes from the Doctrine project, which is nothing but the docblock annotations.

Sample use case from an Inline Entity Form in Drupal 8

Real Entity (node) has been removed from backend when removing contacts field values. Contacts field is an Entity reference field with unlimited cardinality.

By showing this contact with Inline Entity Form - Complex Field Widgets on node edit page.

To avoid this, we need to extend the FieldWidget from an inline entity form - complex. Have a look how to do it programmatically.

Note: Based on the settings, we can control the original entity deletion from the backend. For this, we can use public function settingsForm(array $form, array &$form_state) to add checkbox “Delete referenced node on remove”.

Create a module folder extend_inline_entity_form

Folder Structure: src/Plugin/Field/FieldWidget/ExtendedInlineEntityForm.php

After enabling the module, you can find a similar interface. Here you need to disable “Delete reference node on remove” checkbox to prevent the original node deletion on the backend. Check out the screenshot below for an example.

Field Widgets

Extending an inline entity form - complex - is nothing, but preventing the deletion of an original node from backend when we remove the entity. With the above-mentioned code, we can unset the widget_state[‘delete’] to stop the deletion of an original node from the backend.

For the above code to work, we need to select the “Delete reference node on remove” checkbox to extend an inline entity form - complex. It can be configurable now! We can add this as a patch to drupal.org or we can contribute this as a sub-module to an inline entity form in Drupal. 

That’s it. Hope you can now extend FieldWidgets using an Annotation plugin. If you have any suggestions or queries, please comment down.

Below given is a presentation on extending Field Widgets in Drupal 8.
 

Step-by-step guide to Foundation framework to develop responsive web applications

Creating responsive websites have always remained a challenge for many even I faced similar difficulties in the beginning. Recently, our team came across a situation where we had to design a responsive and beautiful website in Drupal 8 for a media and publishing firm. In order to create such an amazing site, we came up with an idea to use Foundation Framework and yes! it worked.

I have written this blog to help anyone having difficulty in understanding the Foundation framework to develop responsive websites as it is a rising market demand. My idea is that this article will be a "living laboratory" to help you in understanding Foundation from the scratch. The post comprises an intro of Foundation, its features, comparison with Bootstrap, getting started with Foundation. At the end, you can also check out the list of prominent brands using this framework. You may also like to check out our previous blog “An overview of Foundation: A Frontend Framework for Responsive Websites.”

First things first, an Introduction

Foundation is an open source front-end framework that helps in designing responsive websites, apps, and emails. It provides a responsive grid, HTML and CSS UI components, templates, and code snippets, forms, buttons, navigation, other interface components and optional JavaScript extensions to build a responsive site.

Features 

Foundation has been designed for and tested on various browsers and gadgets. The open source framework is a mobile-first that give developers best practices for rapid development. Other features of Foundation - it is consist of 12 column grid system, supports stylized CSS components and RTL (right to left) direction format and compatible with several browsers.

12-Column Grid System:

grid

Credit: Codementor

Media Queries:  

Foundation has three core Breakpoints.

Small  Any Screen up to 639 px.
/* Small only */
@media screen and (max-width: 39.9375em) {...}

Medium Any Screen from 640 px or Wider.
/* Medium only */
@media screen and (min-width: 40em) and (max-width: 63.9375em) {...}

Large Any Screen from 1024 px or Wider.
    /* Large and up */
@media screen and (min-width: 64em) {...}

Compatibility:

In addition to Android browsers, Foundation also supports all the latest versions of Chrome, Firefox, Safari, Opera and Internet Explorer. 


compatability

Credit: Codementor

Widgets:

Below are some widgets & JavaScript features developed in Foundation that Bootstrap does not support.

widgets

Credit: Codementor

Let’s check out some of the differences between Bootstrap and Foundation:

Well, there are many differences and similarities between Bootstrap and Foundation. The main similarity is that they are both great front-end frameworks that can be used to build an awesome application and website.

Comparison

Bootstrap Classes

Foundation Classes
col-xs-* small-*
col-sm-* medium-*
col-md-* large-*
col-lg-*  

CSS3 Preprocessors

CSS3 Preprocessors
LESS SASS(SCSS)
SASS(SCSS)  

RTL (right to left) Format

RTL (right to left) Format
Doesn’t support RTL format Supports RTL format

 

Note: Bootstrap supports both LESS AND SASS (SCSS) Preprocessors, but Foundation supports only SASS (SCSS). Here * represents a number, valid up to 12.

Getting Started with Foundation:

Well, there are many ways to install Foundation, but here I will be discussing two ways: First by CSS and second by using the terminal.

If you are just getting started with CSS then you can download Foundation. Customize it and choose the features you want to have on your website. 

While getting started, you can find a couple of options like Downloading complete Foundation and Download customized Foundation. Select as per your requirement.

After downloading Foundation, your folder structure would look like this. See the screenshot below.

structure

  • app.css  is the custom css file which we can add on our own css
  • foundation.css is the default css implemented by Foundation
  • foundation.min.css  is the minified css 
  • jquery.js is required to provide the Responsive functionality to the website. You just need to have JQuery version 1.10 or higher on your page before the foundation.js library.
  • what-input.js is used to track the current input method. 
  • foundation.js  is used for utilizing core functionality.  
  • app.js  is the custom js file which we can add our js.
  • You need to specify all the above files in your HTML file.
  • You can also install Foundation CLI by using Terminal 
  • sudo npm install --global foundation-cli 
  • Once you've installed the CLI, use the new command to start making a new project:
  •     foundation new

See the screenshot below for an example of Websites designed with Foundation Framework.

sites

Credit: Codementor

Prominent brands using Foundation Framework:     

  1. PBS: Public Broadcasting Service
  2. National Geographic
  3. Washington Post
  4. Mozilla
  5. Cambridge University Press
  6. Official HTC Store (UK)
  7. National Geographic Education
  8. Burger Revolution

 

Using Foundation Framework, we can build responsive websites that will be both unique and beautiful. In this blog, we have explored the basics of Foundation Framework, its overview, features, companies using this framework to design responsive sites as well as how to install customized Foundation using terminal and CSS. This blog is intended to give you a brief intro of the framework and its comparison with Bootstrap. 

Foundation Framework helps in reducing the CSS and JS, so we recommend you to try multiple front-end frameworks to find out what works better for your current project. Based on your comments and suggestions, I plan to add to it, creating additional value. Or maybe it is perfect as-is!

 

An overview of JSON API: A text-based data exchange format

People develop APIs for a variety of reasons, such as to create a tool to facilitate internal processes or an external product for customers or to build a third party tool. And for such purpose JSON comes to the rescue. The open standard format JSON follows shared conventions that help in increasing productivity, take advantage of generalized tooling, and focus on the web applications. 

If you are a developer you must have noticed that there is very little information on the web regarding JSON API when compared to its usability. In this blog post, I will share an overview of JSON API. Let's try and understand what is JSON API? Its advantages over Core REST API, core concepts with working examples.

Note: JSON is an independent component, which can be used by any framework or technology. In our case, Drupal is using its service.

Let’s start with what is JSON API?

JSON or JavaScript Object Notation is an encoding scheme that is designed to eliminate the need for an ad-hoc code for each application to communicate with servers that communicate in a defined way. JSON API module exposes an implementation for data stores and data structures, such as entity types, bundles, and fields.

What are its advantages over Core REST API?

Well, there are lots of advantages, but to mention a few:

  • It reduces both the number of requests and the amount of data transmitted between clients and servers. 

  • Unlike core REST module, JSON API is Zero configuration Drupal module.

  • By enabling the JSON API module, you can immediately gain a full REST API for every type in your Drupal application.

  • JSON API inspects entity type and bundle to provide URLs to access each one of them using the standard HTTP methods, GET, POST, PATCH, and DELETE (we will discuss more on HTTP methods while talking about Document Structure).

  • JSON is not simply a format like JSON or HAL+JSON. The default format appears like:

                       `/jsonapi/{entity_type}/{bundle}/{uuid}?_format=api_json`

  • It also controls which HTTP methods should be used, what HTTP response codes should be returned under specific request, the format of the response body, and the link between resources.

Document Structure:

This section describes the structure of a JSON API document, which is identified by media type. Though the same media type is used for both request and response documents, certain aspects are only applicable to one or the other.

1. Every request/response body must be underneath a single JSON object.

2. data: the information specific to a resource or resources, must live within this top-level object under the data member.

3. type and id: JSON API derives  'type' property from the entity type machine name and bundle machine name.

Note: id in the JSON API are UUID (If we have the same site installed at different places, the nid for the same content mostly be different but UUID remains same on all environments.).

4. attributes and relationship: attributes store values specific to the underlying resource.  relationships represent values that are stored by an entity reference.

Request headers: 

Accept: application/vnd.api+json
Content-Type: application/vnd.api+json

Response Codes:

The JSON API module can respond with the following codes:

200 OK - All successful GET and PATCH requests

201 Created - All successful POST requests (response includes the newly created 
        resource).

204 No Content - All successful DELETE requests

HTTP Methods in Drupal: 

Hypertext Transfer Protocol (HTTP) enables communication between clients and servers. It also defines a set of request methods to perform a certain action for a given resource. In Drupal 8, JSON API supports GET, POST, PATCH and DELETE HTTP methods. 

1. Fetching resources (GET) 

A web request using GET method is made only to retrieve data from a given resource. It specifies different parameters in the URL of the request based on requirements. One can make web request either by using the web browser or any other HTTP client (e.g. Postman chrome app).

To fetch all contents of Article content type, URL should be: example.com/jsonapi/node/article
Sort articles (descending): example.com/jsonapi/node/article?sort=-nid 
Filter out first five contents:example.com/jsonapi/node/article/?page[limit]=5 

Similarly, we can add various filter parameter in the URL based on our requirement.

Note: JSON API lists 50 objects at a time including link to next.

GET-json api

2. Creating new resources (POST)

Submit data to a server to create a new content.

Prerequisites:

  • Make sure HTTP Basic Authentication Drupal core module is enabled on your site.
  • Following headers are required to get correct response:

                                           Accept: application/vnd.api+json
                                           Content-Type: application/vnd.api+json

  • Add basic authentication (credentials to your site)

                                           Authorization: Basic YXBpOmFwaQ==

To create a content named “Hello Article” and body, write the following code in HTTP message body:

URL: example.com/jsonapi/node/article

See the screenshot below for an example

POST-jsonapi

Check if the new content gets created from Drupal site (/admin/content).

3. Updating existing resources (PATCH)

PATCH method submits data to a server to update an existing content.

Prerequisites:

  • Make sure HTTP Basic Authentication Drupal core module is enabled on your site.
  • Following headers are required to get correct response:

                                        Accept: application/vnd.api+json
                                        Content-Type: application/vnd.api+json

  • Add basic authentication (credentials to your site )

                                       Authorization: Basic YXBpOmFwaQ==

To update an article’s title content, write following piece of code in HTTP message body:

URL: example.com/jsonapi/node/article/{{article_uuid}}

PATCH-jsonapi

Verify if the content gets updated in Drupal site(/admin/content).

4. Removing existing resources (DELETE)

DELETE method is used to request the server to delete a specific content.Simply run the following URL to delete any specific article content. 

URL: example.com/jsonapi/node/article/{{article_uuid}}

Prerequisites:

  • Make sure HTTP Basic Authentication Drupal core module is enabled on your site.
  • Following headers are required to get a correct response:

                                      Content-Type: application/vnd.api+json

  • Add basic authentication (credentials to your site )

                                      Authorization: Basic YXBpOmFwaQ==

DELETE-jsonapi

 

Check if the content gets deleted from the Drupal site (/admin/content).

Well, JSON is a way to store information in an organized, easy-to-access manner as it gives us a human-readable collection of data that we can access in a logical manner. Hope this blog will be helpful for you to understand and use JSON. If there is anything you would like to add or see further clarified, please let me know me in comments.

Below given is a presentation on JSON API.

References 

https://www.youtube.com/playlist?list=PLZOQ_ZMpYrZsyO-3IstImK1okrpfAjuMZ 
https://www.drupal.org/docs/8/modules/json-api/json-api 
https://nordicapis.com/3-methods-for-documenting-json-api-services/ 
http://www.json.org/
 

How to Handle Resource Intensive Tasks on Queue With Drupal Queue Worker/API

It's never too late to share something useful with mates. And in this blog, I am going to discuss Queue Worker/API and how to implement it in Drupal 8. After completing this blog cum tutorial you will find yourself at a moderate level of expertise in implementing Queue API.

A couple of months back, we had a situation in one of our project where we had to pull data from a centralized server and update that content to the site almost every day. In order to overcome this scenario, we started using cron, but faced a situation where we were missing few contents because of time out. After a little research and discussion, we (our team) decided to implement queue worker. And yes! It worked.

Let’s start with Queue API.

Queue API enables us to handle a number of tasks at a later stage that means we can insert a number of items in the queue and process them later. The items can be added to the queue by passing an arbitrary data object. And we can run Queue by manually using Drush commands or Cron. Cron is a background process that runs at periodic intervals of time. For an introduction to cron, you can check out our previous blog on “scheduling Automated Tasks in Drupal with Cron”.

So why should we use Queue Worker?

Queue Worker allows you to run queue with time limits. You can also individually queue the tasks at one time compared to Cron that run at the periodic interval of time. In comparison to cron, Queue is more efficient and can handle resource-intensive tasks. The API also allows you to revert the item back to queue if any failure occurs. Most importantly, you can run multiple queues without interrupting other work.

The very basic concept of Queue API is FIFO (First-In-First-Out).

Check out an example of the above-mentioned scenario.

Here we will create a Queue Worker that will fetch data from centralized repo using API and create an article on our existing site. This process will run automatically based on cron.

See the screenshot below for an example of the folder structure that will contain required files and their respective location:

Article_queue_folder_structure

Module Setup:

article_queue.info.yml

In Drupal 8, the menu system has been replaced by the routing system.

article_queue.routing.yml

ArticleSettingsForm.php

Create the Configuration form to store API link and their credentials. After entering Connector Site URL, Site User Name and Site Password, save the configuration.

ConfigForm

Let's see how we can use cron to add data to Queue.

In a hook_cron method, we will be defining a method to fetch data from API and adding to queue. 

First, create a QueueFactory object from the service container statically and use this object to instantiate a queue object of article_queue_processor.

Here we call create Item method to insert items in the queue. Once cron runs it will insert all items fetched from API in the process queue.

article_queue.module

In this example, we will be using ultimate cron for processing the cron. We need to set up we need to create a config file.

ultimate_cron.job.node_queue_worker.yml  

Need to place inside install folder. If the module is already enabled, we need to import this file using configuration management interface.

Now create a class to call processItem() that will process the queue.

ArticleQueueProcessor.php

In this method, we are going to define the processItem() method to create a node from the data received in API. 

Once cron run is completed, an article will be created on the site. 

To check this manually, we can navigate to “admin/config/system/cron/jobs”  and run the cron “Article Queue Worker”. After cron run navigate to “admin/config/system/queue-ui” and select “Task Worker: Node” and run batch process. Once the process is over your content will be created on the site.

That’s it!

See how easily you can deploy Queue Worker/API in Drupal to handle a number of resource intensive tasks at a later point of time. With Queue Worker, items can be added by passing an arbitrary data object. Also, using this API, you can easily manage your resource intensive tasks on your website. 

Hope you can now implement Queue Worker/API in Drupal. If you have any suggestions or queries please comment down let me try to answer.
 

Restricting access for anonymous users on a Drupal website running on Apache

Hello folks! Here is another blog that will help you to restrict anonymous or unwanted users entry on your web application when it is in development mode on the server. You will be wondering why did I come up with such an idea? And how it will help you out? This is what we will be discussing in this post. Quite a times we ignore to restrict anonymous entries due to some data integrity and that results in spoiling our site.

Let me tell you how? 

Recently, one of my friends has started working on his Drupal website from the scratch and he wants to stop the anonymous entries whether it's a new registration or anonymous view or comments as he is working on some serious stuff. Further, he doesn’t want to spoil the site from unknown users. And for this, he decided the restrict web application to an anonymous user. You may also like to check out our previous post where we have explained how to secure user’s private data from unauthorized access by enabling SSL on web server.

Note: It will be open up for generic users, who all are the part of the ongoing development with the help of Authentication Prompt.

Check out some of the ways that will help you to tackle the similar situation: 

  • To restrict registration - Don’t allow anonymous user registration

  • To restrict login - If you follow first

  • Anonymous Comment - Turn off comment system 

The above-mentioned ways are alternatives to prevent anonymous users from getting access to your site. When you limit access to your website, you need to allow site access only to those users who have correct admin credentials. And they are only allowed to grant the access. This can be accomplished using .htaccess & .htpasswd.
 
I guess you won’t be surprised of hearing these two names as you might have seen these residing in the root directory of your server.

.htaccess: A configuration file in '.htaccess' is used to run a web server like Apache. When a .htaccess file placed in the root directory of your website, it gets loaded through Apache, and then .htaccess file gets executed by Apache web server.

The .htaccess file is used to alter the configuration, enable or disable additional functionality and features of the Apache web server. Also, these files provide basic redirection, content password protection or image hotlink prevention. This method is called htaccess password protection or htaccess authentication.
 

htaccess location in root filder

 

Creating a htaccess file

You can create a .htaccess file by entering few lines of code that needs to be read by Apache. Further, to protect the site you need to write code that will be read by Apache before directing from the server to the end users.

AuthUserFile  full path of .htpasswd
AuthType Basic
AuthName "write some description here"
Require  valid-user

The AuthName parameter defines the title of the password entry box while login. Remember, here, the declaration is important.

The AuthType is a method used to authenticate the user. The common method is Basic, which is implemented by mod_auth_basic. The basic authentication sends credential from browser to server.

AuthUserFile: It is a full path of .htpasswd file.

Require: It provides authorization to set up the user, who is allowed to access the area on the server.

Note: We can give access only to specific users and all other users are supposed to be listed in .htpassword.

Only to specific user

require user  parameter in .htaccess for individual access

Below-mentioned source codes are from my local .htaccess that allows access to specific users

AuthUserFile C:\xampp\htdocs\.htpasswd
AuthType Basic
AuthName "My Secret Folder" 
require user xaiwant

All user listed in .htpasswd

require valid-user parameter in .htacess for all user access.

Following source codes are from my local .htpassword that give access to all users. 

xaiwant:$apr1$zx523D3t$VXU..vXqErSd1wOlSI9k41
want:$apr1$d7eWG8M.$PVnGYLZwuuzIXRXNxQ4Be0

With a password, the .htaccess file can protect your website, including all files in the folder and sub-folder too. Not to mention if you want to protect your website from unauthorized access, you have to place a .htaccess file in root folder.

AuthUserFile C:\xampp\htdocs.htpasswd\.htpasswd
AuthType Basic
AuthName "Drupal 7 Instance"
Require user xaiwant

 

.htpasswd: htpasswd file is used to store usernames and password for basic authentication of HTTP users. This file is called as Apache htpasswd aka Apache password file.

In case you find any difficulty while login to the prompt, then it must be htpasswd access issue (Read/Write) due to permission and same is likely to throw an error.
 
Resources available from Apache can be restricted to users by using the files created by htpasswd. Files managed by htpasswd may contain a mixture of different encoding types of passwords.

Creating a password file

Create a text file that store username and password separated by colon [:]. The password should be in encrypted form, which means you can utilize online web tools to generate them. Here I am sharing one of the URLs.

http://www.htaccesstools.com/htpasswd-generator/

Below is the encrypted credential of one of my local instance:

xaiwant:$apr1$zx523D3t$VXU..vXqErSd1wOlSI9k41

Copy and paste your encrypted credentials and then save the file. Place the folder out of your directory so that nobody can view the file. 

Similarly, you can also create a file name with your choice like xyz. It’s always suggested to change the filename so that it can’t be guessed by Hackers. FYI, Apache provides better file security in terms of web-based access to those files which starts with .ht. 

Still, if you want to know more then please visit http://www.apache.org/ or Google it for more valuable and authentic information.

To add password protection, you need to: 

  • Create a txt file .htpasswd on your server, which will store your username and password.

  • Create a special file, called .htaccess, in the folder you want to protect.
     

That's it! Now let's take a look at how to do each step.

Note: Don’t forget to Restart your server after making all the required changes.

Now hit the URL in the browser and you will get a prompt box asking for username and password. Enter the corrected credential and you are in.

Authentication Pop up

 

Note: You can access the folder n number of times on the server or on the local machine till your browser is open. Once you restart the browser, you will be asked for login credentials again with prompt box.

In case you haven’t been asked for credential then you need to check whether: 

  1. File path is correct or not.
  2. File is accessible or not. Related to permission.
  3. Entered credentials are correct or not.

There you have it! I guess this is the easiest and simplest way to set up the restriction on Drupal website that is running on Apache server. By using “.htaccess” and “.htpasswd”, you can easily limit your site from anonymous entries whether it’s a new registration or anonymous view or comments.

The goal of our blog is to bring you valuable information to help you grow your business. We hope you enjoy this post! We invite you to comment below or ask any questions you may have.
 

Managing Drupal 8 applications remotely using Drush aliases

Have you ever thought that your business needs to make sure that your web application has a quick release in order to sustain the long race. This sort of has become easy to manage by continuous development & continuous integration using Drush & Drush Aliases. Drupal web development is one such place where multiple command-line interface (CLI) tools are available to make developer’s life easy, and among them, the two important things are Drush and Drupal Console.

In this blog, we will take a brief look at Drush & Drush Aliases and how it can make developer’s tedious manual web development tasks easy by offering various commands to replace the manual steps. You may also like to check out our previous blog where we have discussed how to write the custom Drush Commands in Drupal 8.

First thing first, what is Drush?

Drush is a Command Line Shell Interface, which provides its own dictionary of commands to interact with Drupal modules/ themes/ profiles. For instance, If you want to enable the Drupal project (modules) on your website then all you need to do is execute the following command in your CLI.

Drush command

How to install Drush in Drupal 8?

The easiest way to install Drush on Drupal 8 instance is by using Composer. Just hit the below-mentioned command in your CLI and you are done with installation. Learn more about installing Drupal with drush on our previous blog.

composer command

 

What are Drush Aliases?

An ‘Alias’, in general, is nothing but a small representation/acronym for something bigger. In case of Drush, the ‘Aliases’ are a short form of Drush Commands. Quite often when we have to ssh on our remote servers to perform different operations, the process becomes a tedious task and even more tedious when we have multiple instances. This is where Drush Aliases comes to our rescue.

Drush

Here you have three instances “Dev > Stage > Production”, remember, all the three instances can be configured by Aliases so that Drush can perform its tasks on the remote server(s) from your local server.

Some of the common tasks can be ‘Database Sync’, ‘Files Sync’, ‘Cache Clearance’, ‘Module Operations’, ‘importing and exporting the configuration’ e.t.c.

How to set up local Aliases?

  1. Create an alias file in .drush folder of your home directory(~/.drush) to make Drush aliases (short commands) accessible from anywhere. Drush alias file must be named similar to the below convention.                                           

sitename command

 

Make sure your newly created alias file contains below snippet.

Similarly, you can create alias file for remote servers as well.

      2. The following snippet, when placed in your alias file, will automatically create aliases for all of your projects.

Note:  You can add $local_sites path and /docroot as per your local configuration.

If you are using Acquia and Pantheon as a remote server, you can follow the below-mentioned steps to set up the aliases on your local machine.

Acquia and Drush Aliases

  1. Sign in to the Acquia Cloud interface using your credentials.
  2. Click your name in the upper left corner and then click Edit profile.
  3. On the Profile page, click Credentials and then enter your Acquia password.
  4. Under Drush integration, click Download Drush aliases
  5. Extract the downloaded archive file into $HOME:  
  6. tar -C $HOME -xf $HOME/Downloads/mysite.tar.gz or you can place mysite.aliases.drushrc.php file into .drush folder manually.
  7. Now go to your terminal and run drush sa command to check aliases.

Pantheon and Drush Aliases

  1. Sign in to Pantheon dashboard using your account details.
  2. On the home page you will find Drush Aliases, click on it to download the file.
  3. Now you can place that file inside .drush folder.

Drush commands - A fast and easy-to-use command reference for Drush users.

  • $ drush @staging updatedb - This command is used to run pending updates on staging site.

  • $ drush rsync @staging:%files/ @live:%files - You can use this command to Synchronize staging files to production.

  • $ drush sql-sync --structure-tables-key=custom @live @local - This command is used to Synchronize database from production to local, excluding the cache table.

  • $ drush cex @dev, $ drush cim @live - This command is used to export and import the configuration.

Since we did not cover all Drush Commands and their usage, it will be helpful for you if you can check out Drush.org and Drush Commands.

Okay so now you know what is Drush and Drush Aliases and how it makes life easier for both Drupal developers and site administrators. We recommend you to try out Drush Commands to find out what works well for your current project or requirements. But there are many more advanced commands that are really worth learning.

If you have any suggestions or queries please comment down let me try to answer.

Happy Drushing!

How to create custom token to be used in default mail message template in Drupal 8

Sometimes we need to do similar coding in different places, such as for account settings email templates (Welcome email template, Forget password email template, from UI to get the same results. In this scenario, it's always suggested to create a custom token and use that in different types of email templates (account setting email templates) from Drupal UI in [token:type] format.

In our previous blog, we explored about creating custom tokens in Drupal 7 and in this, we will take a brief look at what is token? How to create it in Drupal 8?

In the vein of first thing first, what is token?

Tokens are very small bits of data or text that we use or place into larger documents via simple placeholders, like [type:token], to create custom tokens. The Token module provides an API that is used to provide tokens for all other core or contributed module in a bunch and provide browsing facility for all available tokens in UI.

When we install Drupal in our system, some Drupal core modules has available tokens that we can use in our code as well as in account setting email templates. But as a beginner in Drupal 8, It's really difficult to find out which core tokens are available and to find this, we use Token module. 

See the screenshot below for an example of available tokens.

Available token

Requirements

Sometimes, we need to create custom tokens as per the project's requirement and for that, we need two hooks, namely

1. hook_token_into()
2. hook_tokens()

hook_token_into(): This hook will allow you to provide the information of types of tokens and placeholders of tokens.
hook_tokens(): This hook provides replacement values of tokens. Some parameters are needed for this hook. Click here to check the parameters.

Here, I am creating a simple token to print “Hello World!” text as token replacement parameter.

Here we are going to see how to create a simple custom token in Drupal 8.

Step 1: Create a custom module for this custom token. Here I am creating a custom module name custom_module that contains the below-mentioned files.

custom_token.info.yml
custom_token.module
custom_token.tokens.inc

Step 2: Create a custom_token.info.yml file

Step 3: Create “custom_token.module file”.

Step 4: Create “custom_token.tokens.inc” file. Here, I have added all the custom codes for custom token.

Note: You can put both hooks and require bubble metadata class in “custom_token.module” file directly without creating any .inc file.

Ones you create and install this module, you will find the newly created custom token is displaying into the Browse available tokens section.

Custom token in browser list

Now you can use this newly created custom token in Account setting Email section as the below format:

    [custom_token_type:custom_token_name]

See the screenshot below for an example

Custom token apply in template

Okay, so now you know what is custom Token and how to create it? You're also convinced that you've got something new to say about your subject. Go ahead! Try custom Token on your website and see what it can do for you. I invite you to comment below or ask any questions you may have. 

Below given is a presentation on creating custom tokens in Drupal 8.

 

Download the Drupal Guide
Enter your email address to receive the guide.
get in touch