Creating custom annotations in Symfony 3

I’m currently building a backend API for a site which requires requests to be signed with an access token header. My problem is, the default @Security annotation feature doesn’t fit with our requirements, because we don’t actually have any users on our site! We actually consume our clients SOAP service, which is where all the important data is stored.

Currently, in a typical controller action, I have the following code:

/**
 *@Rest\Put("/auth/change-password")
 */
public function changePasswordAction(Request $request)
{
 $tokenId = $paramFetcher->get('token');
 $tokenSvc = $this->get('Our\SuperbBundle\Service\TokenService');

 // Check the token exists and is valid
 // throws exception if not found or expired
 $tokenSvc->findToken($tokenId); 
 // actual change password logic here
}

We are going to have several controllers with multiple calls, so we don’t want to add this in every single controller action, so we will create our own custom annotation.

<?php

namespace Our\CustomerZoneBundle\Annotation;

/**
 * Class SecureToken
 * @package Our\CustomerZoneBundle\Annotation
 * @Annotation
 */
class SecureToken
{
}

The next thing we do is create an Annotation listener, which I’ll explain below:

<?php

namespace Our\CustomerZoneBundle\EventListener;

use Doctrine\Common\Annotations\AnnotationReader;
use Doctrine\Common\Util\ClassUtils;
use Our\CustomerZoneBundle\Annotation\SecureToken;
use Our\CustomerZoneBundle\Model\Exception\TokenException;
use Our\CustomerZoneBundle\Service\TokenService;
use ReflectionClass;
use ReflectionObject;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\RequestStack;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\Event\FilterControllerEvent;
use Symfony\Component\HttpKernel\Exception\AccessDeniedHttpException;

class SecureTokenAnnotationListener
{
    /** @var AnnotationReader $reader */
    protected $reader;

    /** @var RequestStack $requestStack */
    protected $requestStack;

    /** @var TokenService $tokenService */
    protected $tokenService;

    /**
     * SecureTokenAnnotationListener constructor.
     * @param AnnotationReader $reader
     * @param RequestStack $stack
     * @param TokenService $tokenService
     */
    public function __construct(AnnotationReader $reader, RequestStack $stack, TokenService $tokenService)
    {
        $this->reader = $reader;
        $this->requestStack = $stack;
        $this->tokenService = $tokenService;
    }

    /**
     * @param FilterControllerEvent $event
     */
    public function onKernelController(FilterControllerEvent $event)
    {
        $controller = $event->getController();
        /*
         * $controller passed can be either a class or a Closure.
         * This is not usual in Symfony2 but it may happen.
         * If it is a class, it comes in array format
         *
         */
        if (!is_array($controller)) {
            return;
        }

        /** @var Controller $controllerObject */
        list($controllerObject, $methodName) = $controller;

        $request = $this->requestStack->getCurrentRequest();
        $cookies = $request->cookies;
        $mkzCookie = $cookies->get('mkz');

        // Override the response only if the annotation is used for method or class
        if ($this->hasSecureTokenAnnotation($controllerObject, $methodName)) {
            try {
                if (!$mkzCookie) {
                    throw new TokenException(TokenException::ERROR_NOT_FOUND, 404);
                }
                $this->tokenService->findToken($mkzCookie);
            } catch (TokenException $e) {
                throw new AccessDeniedHttpException($e->getMessage(), $e);
            }
        }
    }

    /**
     * @param Controller $controllerObject
     * @param string $methodName
     * @return bool
     */
    private function hasSecureTokenAnnotation(Controller $controllerObject, string $methodName) : bool
    {
        $tokenAnnotation = SecureToken::class;

        $hasAnnotation = false;

        // Get class annotation
        // Using ClassUtils::getClass in case the controller is an proxy
        $classAnnotation = $this->reader->getClassAnnotation(
            new ReflectionClass(ClassUtils::getClass($controllerObject)), $tokenAnnotation
        );

        if ($classAnnotation) {
            $hasAnnotation = true;
        }

        // Get method annotation
        $controllerReflectionObject = new ReflectionObject($controllerObject);
        $reflectionMethod = $controllerReflectionObject->getMethod($methodName);
        $methodAnnotation = $this->reader->getMethodAnnotation($reflectionMethod, $tokenAnnotation);

        if ($methodAnnotation) {
            $hasAnnotation = true;
        }

        return $hasAnnotation;
    }
}

Our event listener uses the annotation reader class, and takes the request object. When the event triggers, the onKernelController() method is called, where we get the Access Token cookie from the request. Then it checks for the @SecureToken annotation, which can either cover an entire class, or can be set in individual method docblocks.

Next, we hook up the service autowiring in the config services.yml:

our.event_listener.secure_token:
    class: Our\CustomerZoneBundle\EventListener\SecureTokenAnnotationListener
    autowire: true
    tags:
        - { name: kernel.event_listener, event: kernel.controller }

Symfony autowiring is pretty cool. the autowire true key allows Symfony to take care of object instantiation, so we don’t even need to fetch the annotation reader or request for the constructor!

Finally, we add our annotation to our method, and remove our check from inside the method:

/**
 * @Rest\Put("/auth/change-password")
 * @SecureToken
 */
public function changePasswordAction(Request $request)
{
    // actual change password logic here
}

Now using whatever HTTP client you like (POSTman 😉 !) when we add our Access Token cookie header, the check happens automatically! Removing the cookie now gives us a 403 response! Success! Have fun! 😀

Advertisements

Apigility Oauth2 Clients

This is mainly for my own notes (I’ll update), but I’m going to use OAuth2 for an API I’ll be building using Apigility. First thing I did was add a client into the oauth_clients table. the password should be bcrypted, and as I’m doing a server to server situation (my web site (the client) to the api) i type authorization_code into the grant_type. In the redirect_url put /oauth/receivecode

Next up I had issues getting the Postman client OAuth setup to work correctly, so I decided to use an awesome cli alternative, HTTPie, which can be installed by sudo apt-get install httpie.Run the following command to request an authorization code:

http --verify no -f POST https://api.awesome.del/oauth/authorize client_id=testclient response_type=code  redirect_uri=/oauth/receivecode state=xyz authorized=yes

The –verify no option stops it freaking out if the SSL certificate (OAuth2 MUST be used on port 443!) is a self signed one. We pass in the client_id, response_type, a redirect_uri, state can be anything (available to use for csrf?), and authorized=yes is the form variable on apigilities authorization page. You should get something like this:

HTTP/1.1 302 Found
Connection: close
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Tue, 28 Jul 2015 05:36:56 GMT
Location: /oauth/receivecode?code=668d8b8d8ac4442fdf51419bf89c33f8b49f9971&state=xyz
Server: Apache/2.4.12 (Ubuntu)

As you can see, $_GET[‘code’] is there. Once we have authorised a client to the api, it adds a row to oauth_authorization_codes, with the client, callback, and an expiry date not long at all after being generated. Using the code variable, you run the following command:

http --verify no -f POST https://api.awesome.del/oauth client_id=testclient client_secret=testpass grant_type=authorization_code  redirect_uri=/oauth/receivecode code=668d8b8d8ac4442fdf51419bf89c33f8b49f9971

If all is going well you should see this:

HTTP/1.1 200 OK
Cache-Control: no-store
Connection: close
Content-Type: application/json
Date: Tue, 28 Jul 2015 05:43:02 GMT
Pragma: no-cache
Server: Apache/2.4.12 (Ubuntu)
Transfer-Encoding: chunked

{
    "access_token": "a920d6e7d85a2eb6225c07efb17c0f1fb7b5e9e6", 
    "expires_in": 3600, 
    "refresh_token": "fea627801b8d1d4fce095c7cef6dde7c0d3b5850", 
    "scope": null, 
    "token_type": "Bearer"
}

Now that you have the access_token and refresh_token, we can start calling the sections of our api that we secure up! Currently I have a /ping endpoint that returns the server time, just for testing connectivity. In the Apigility Panel, click on the name of the API, and set the authentication type to your OAuth2 one. Then in your api service (ping in my example), in the authorization tab, click GET checkbox to run authorization on GET /ping. Now if you access /ping without authorisation, you should see the following:

{
  "type": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html",
  "title": "Forbidden",
  "status": 403,
  "detail": "Forbidden"
}

So we need to access with our access_token. At this point I ran into an annoying problem, which to save you a lot of hassle, was this; that Apache 2.4 is set by default to strip out the Authorization Header! Amazingly, my answer was in my own post about getting Basic HTTP Authentication working in puPHPet! You need to set in your vhosts (or in my below example, the puphpet config.yaml) a setenvif:

setenvif:           
     - 'Authorization "(.*)" HTTP_AUTHORIZATION=$1'

Reprovision your puppet box or restart apache if you edited your vhosts file. No we should be able to accept requests with an access_token:

http --verify no GET https://api.awesome.del/ping "Authorization:Bearera920d6e7d85a2eb6225c07efb17c0f1fb7b5e9e6"

And you should see:

HTTP/1.1 200 OK
Connection: close
Content-Type: application/json; charset=utf-8
Date: Tue, 28 Jul 2015 21:06:10 GMT
Server: Apache/2.4.12 (Ubuntu)
Transfer-Encoding: chunked

{
    "ack": "2015-07-28 21:06:12"
}

So now we have resources which can be set public, and which can require authorisation. I’ll update this once I learn some more about the oauth_scopes. Have fun 🙂

json_encode Problem? Probably not UTF-8 then

I had this weird problem today where my API was working fine on the development machine, but on my production server I was getting a blank 200 text/html page.

It turned out that the array that was being changed into JSON contained certain characters which were not UTF-8, meaning that a var_dump($json) returned bool : false.

I found a nice class on github which has some static methods for doing a bunch of utf-8 character encoding fixes. Feeding my array into the fixUtf8() method sorted everything!

https://github.com/neitanod/forceutf8

<?php
/*
Copyright (c) 2008 Sebastián Grignoli
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of copyright holders nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
“AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL COPYRIGHT HOLDERS OR CONTRIBUTORS
BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
*/
/**
* @author “Sebastián Grignoli” <grignoli@framework2.com.ar>
* @package Encoding
* @version 1.2
* @license Revised BSD
*/
namespace ForceUTF8;
class Encoding {
  protected static $win1252ToUtf8 = array(
        128 => \xe2\x82\xac,
        130 => \xe2\x80\x9a,
        131 => \xc6\x92,
        132 => \xe2\x80\x9e,
        133 => \xe2\x80\xa6,
        134 => \xe2\x80\xa0,
        135 => \xe2\x80\xa1,
        136 => \xcb\x86,
        137 => \xe2\x80\xb0,
        138 => \xc5\xa0,
        139 => \xe2\x80\xb9,
        140 => \xc5\x92,
        142 => \xc5\xbd,
        145 => \xe2\x80\x98,
        146 => \xe2\x80\x99,
        147 => \xe2\x80\x9c,
        148 => \xe2\x80\x9d,
        149 => \xe2\x80\xa2,
        150 => \xe2\x80\x93,
        151 => \xe2\x80\x94,
        152 => \xcb\x9c,
        153 => \xe2\x84\xa2,
        154 => \xc5\xa1,
        155 => \xe2\x80\xba,
        156 => \xc5\x93,
        158 => \xc5\xbe,
        159 => \xc5\xb8
  );
    protected static $brokenUtf8ToUtf8 = array(
        \xc2\x80 => \xe2\x82\xac,
        \xc2\x82 => \xe2\x80\x9a,
        \xc2\x83 => \xc6\x92,
        \xc2\x84 => \xe2\x80\x9e,
        \xc2\x85 => \xe2\x80\xa6,
        \xc2\x86 => \xe2\x80\xa0,
        \xc2\x87 => \xe2\x80\xa1,
        \xc2\x88 => \xcb\x86,
        \xc2\x89 => \xe2\x80\xb0,
        \xc2\x8a => \xc5\xa0,
        \xc2\x8b => \xe2\x80\xb9,
        \xc2\x8c => \xc5\x92,
        \xc2\x8e => \xc5\xbd,
        \xc2\x91 => \xe2\x80\x98,
        \xc2\x92 => \xe2\x80\x99,
        \xc2\x93 => \xe2\x80\x9c,
        \xc2\x94 => \xe2\x80\x9d,
        \xc2\x95 => \xe2\x80\xa2,
        \xc2\x96 => \xe2\x80\x93,
        \xc2\x97 => \xe2\x80\x94,
        \xc2\x98 => \xcb\x9c,
        \xc2\x99 => \xe2\x84\xa2,
        \xc2\x9a => \xc5\xa1,
        \xc2\x9b => \xe2\x80\xba,
        \xc2\x9c => \xc5\x93,
        \xc2\x9e => \xc5\xbe,
        \xc2\x9f => \xc5\xb8
  );
  protected static $utf8ToWin1252 = array(
       \xe2\x82\xac => \x80,
       \xe2\x80\x9a => \x82,
       \xc6\x92 => \x83,
       \xe2\x80\x9e => \x84,
       \xe2\x80\xa6 => \x85,
       \xe2\x80\xa0 => \x86,
       \xe2\x80\xa1 => \x87,
       \xcb\x86 => \x88,
       \xe2\x80\xb0 => \x89,
       \xc5\xa0 => \x8a,
       \xe2\x80\xb9 => \x8b,
       \xc5\x92 => \x8c,
       \xc5\xbd => \x8e,
       \xe2\x80\x98 => \x91,
       \xe2\x80\x99 => \x92,
       \xe2\x80\x9c => \x93,
       \xe2\x80\x9d => \x94,
       \xe2\x80\xa2 => \x95,
       \xe2\x80\x93 => \x96,
       \xe2\x80\x94 => \x97,
       \xcb\x9c => \x98,
       \xe2\x84\xa2 => \x99,
       \xc5\xa1 => \x9a,
       \xe2\x80\xba => \x9b,
       \xc5\x93 => \x9c,
       \xc5\xbe => \x9e,
       \xc5\xb8 => \x9f
    );
  static function toUTF8($text){
  /**
* Function Encoding::toUTF8
*
* This function leaves UTF8 characters alone, while converting almost all non-UTF8 to UTF8.
*
* It assumes that the encoding of the original string is either Windows-1252 or ISO 8859-1.
*
* It may fail to convert characters to UTF-8 if they fall into one of these scenarios:
*
* 1) when any of these characters: ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß
* are followed by any of these: (“group B”)
* ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶•¸¹º»¼½¾¿
* For example: %ABREPRESENT%C9%BB. «REPRESENTÉ»
* The “«” (%AB) character will be converted, but the “É” followed by “»” (%C9%BB)
* is also a valid unicode character, and will be left unchanged.
*
* 2) when any of these: àáâãäåæçèéêëìíîï are followed by TWO chars from group B,
* 3) when any of these: ðñòó are followed by THREE chars from group B.
*
* @name toUTF8
* @param string $text Any string.
* @return string The same string, UTF8 encoded
*
*/
    if(is_array($text))
    {
      foreach($text as $k => $v)
      {
        $text[$k] = self::toUTF8($v);
      }
      return $text;
    } elseif(is_string($text)) {
      $max = strlen($text);
      $buf = “”;
      for($i = 0; $i < $max; $i++){
          $c1 = $text{$i};
          if($c1>=\xc0){ //Should be converted to UTF8, if it’s not UTF8 already
            $c2 = $i+1 >= $max? \x00 : $text{$i+1};
            $c3 = $i+2 >= $max? \x00 : $text{$i+2};
            $c4 = $i+3 >= $max? \x00 : $text{$i+3};
              if($c1 >= \xc0 & $c1 <= \xdf){ //looks like 2 bytes UTF8
                  if($c2 >= \x80 && $c2 <= \xbf){ //yeah, almost sure it’s UTF8 already
                      $buf .= $c1 . $c2;
                      $i++;
                  } else { //not valid UTF8. Convert it.
                      $cc1 = (chr(ord($c1) / 64) | \xc0);
                      $cc2 = ($c1 & \x3f) | \x80;
                      $buf .= $cc1 . $cc2;
                  }
              } elseif($c1 >= \xe0 & $c1 <= \xef){ //looks like 3 bytes UTF8
                  if($c2 >= \x80 && $c2 <= \xbf && $c3 >= \x80 && $c3 <= \xbf){ //yeah, almost sure it’s UTF8 already
                      $buf .= $c1 . $c2 . $c3;
                      $i = $i + 2;
                  } else { //not valid UTF8. Convert it.
                      $cc1 = (chr(ord($c1) / 64) | \xc0);
                      $cc2 = ($c1 & \x3f) | \x80;
                      $buf .= $cc1 . $cc2;
                  }
              } elseif($c1 >= \xf0 & $c1 <= \xf7){ //looks like 4 bytes UTF8
                  if($c2 >= \x80 && $c2 <= \xbf && $c3 >= \x80 && $c3 <= \xbf && $c4 >= \x80 && $c4 <= \xbf){ //yeah, almost sure it’s UTF8 already
                      $buf .= $c1 . $c2 . $c3;
                      $i = $i + 2;
                  } else { //not valid UTF8. Convert it.
                      $cc1 = (chr(ord($c1) / 64) | \xc0);
                      $cc2 = ($c1 & \x3f) | \x80;
                      $buf .= $cc1 . $cc2;
                  }
              } else { //doesn’t look like UTF8, but should be converted
                      $cc1 = (chr(ord($c1) / 64) | \xc0);
                      $cc2 = (($c1 & \x3f) | \x80);
                      $buf .= $cc1 . $cc2;
              }
          } elseif(($c1 & \xc0) == \x80){ // needs conversion
                if(isset(self::$win1252ToUtf8[ord($c1)])) { //found in Windows-1252 special cases
                    $buf .= self::$win1252ToUtf8[ord($c1)];
                } else {
                  $cc1 = (chr(ord($c1) / 64) | \xc0);
                  $cc2 = (($c1 & \x3f) | \x80);
                  $buf .= $cc1 . $cc2;
                }
          } else { // it doesn’t need convesion
              $buf .= $c1;
          }
      }
      return $buf;
    } else {
      return $text;
    }
  }
  static function toWin1252($text) {
    if(is_array($text)) {
      foreach($text as $k => $v) {
        $text[$k] = self::toWin1252($v);
      }
      return $text;
    } elseif(is_string($text)) {
      return utf8_decode(str_replace(array_keys(self::$utf8ToWin1252), array_values(self::$utf8ToWin1252), self::toUTF8($text)));
    } else {
      return $text;
    }
  }
  static function toISO8859($text) {
    return self::toWin1252($text);
  }
  static function toLatin1($text) {
    return self::toWin1252($text);
  }
  static function fixUTF8($text){
    if(is_array($text)) {
      foreach($text as $k => $v) {
        $text[$k] = self::fixUTF8($v);
      }
      return $text;
    }
    $last = “”;
    while($last <> $text){
      $last = $text;
      $text = self::toUTF8(utf8_decode(str_replace(array_keys(self::$utf8ToWin1252), array_values(self::$utf8ToWin1252), $text)));
    }
    $text = self::toUTF8(utf8_decode(str_replace(array_keys(self::$utf8ToWin1252), array_values(self::$utf8ToWin1252), $text)));
    return $text;
  }
  static function UTF8FixWin1252Chars($text){
    // If you received an UTF-8 string that was converted from Windows-1252 as it was ISO8859-1
    // (ignoring Windows-1252 chars from 80 to 9F) use this function to fix it.
    return str_replace(array_keys(self::$brokenUtf8ToUtf8), array_values(self::$brokenUtf8ToUtf8), $text);
  }
  static function removeBOM($str=“”){
    if(substr($str, 0,3) == pack(“CCC”,0xef,0xbb,0xbf)) {
      $str=substr($str, 3);
    }
    return $str;
  }
  public static function normalizeEncoding($encodingLabel)
  {
    $encoding = strtoupper($encodingLabel);
    $enc = preg_replace(‘/[^a-zA-Z0-9\s]/’, , $encoding);
    $equivalences = array(
        ‘ISO88591’ => ‘ISO-8859-1’,
        ‘ISO8859’ => ‘ISO-8859-1’,
        ‘ISO’ => ‘ISO-8859-1’,
        ‘LATIN1’ => ‘ISO-8859-1’,
        ‘LATIN’ => ‘ISO-8859-1’,
        ‘UTF8’ => ‘UTF-8’,
        ‘UTF’ => ‘UTF-8’,
        ‘WIN1252’ => ‘ISO-8859-1’,
        ‘WINDOWS1252’ => ‘ISO-8859-1’
    );
    if(empty($equivalences[$encoding])){
      return ‘UTF-8’;
    }
    return $equivalences[$encoding];
  }
  public static function encode($encodingLabel, $text)
  {
    $encodingLabel = self::normalizeEncoding($encodingLabel);
    if($encodingLabel == ‘UTF-8’) return Encoding::toUTF8($text);
    if($encodingLabel == ‘ISO-8859-1’) return Encoding::toLatin1($text);
  }
}


			

Manually compiling PHP modules successfully

This look familiar?

PHP Warning:  PHP Startup: memcached: Unable to initialize module
Module compiled with module API=20090626
PHP    compiled with module API=20100525
These options need to match

I don’t know about you, but i like to be up to date! My PHP is on 5.5, and I had to install some modules. But sometimes, old versions can rear their ugly head, and cause all manner of grief. Package managers do a good job to take care of all this for you, but sometimes they just don’t work. Leaving you to compile yourself! So lets do it! I’m going to install memcached, and then the imagick libraries (now i know what i’m doing!)

I’m doing this on a CentOS 6 server, but as we are doing the old skool way of compiling etc, this should work on any other flavour of Linux, or indeed Mac OS X.

First step is to download your .tar.gz  then unzip it with tar -zxvf file.tar.gz and change into the folder.
Bring up a web page displaying your servers php.ini. You are looking for the version of PHP API, and the extension_dir.
In your terminal, cd into the module source code folder, and type phpize.

If when you check the API versions , they are different from your php.ini, then an old version of php is being used in the terminal, and your module will not work! In this case, you need to get it to use the correct phpize.

type 'which phpize' to find out where the offending file is. (mine was /usr/bin/phpize)

My PHP appeared to be in /usr/local, so I tried running /usr/local/phpize. The API’s matched. So then I did the following:

mv /usr/bin/phpize /usr/bin/phpize-old
 ln -s /usr/local/bin/phpize /usr/bin/phpize

Half way there! We need to do the same for php-config

mv /usr/bin/php-config /usr/bin/php-config-old
 ln -s /usr/local/bin/php-config /usr/bin/php-config

Now you have done that, installation should be trivial, and work as per loads of tutorial/instrruction pages on the web.

./configure
 make
 make install

Finally edit your php.ini and add ‘extension = memcached.so’ (or whatever module you compiled), and restart your apache server!

EDIT : you may need to run ‘phpize –clean’ if it is still compiling with the older stuff from within the modules source folder