Documentation

CurlFetcher extends WebFetchApi
in package

Class CurlFetcher

Simple curl class to fetch a web page. Properly redirects even with safe mode and basedir restrictions. Can provide simple post options to a page.

Load class

Initiate as

$fetch_data = new CurlFetcher();

Optionally pass an array of curl options and redirect count

$fetch_data = new CurlFetcher(array(CURLOPT_SSL_VERIFYPEER => 1), 5);

Make the call

Fetch a page

$fetch_data->request('https://www.simplemachines.org');

Post to a page providing an array

$fetch_data->request('https://www.simplemachines.org', array('user' => 'name', 'password' => 'password'));

Post to a page providing a string

$fetch_data->fetch('https://www.simplemachines.org', parameter1&parameter2&parameter3);

Get the data

Just the page content

$fetch_data->result('body');

An array of results, body, header, http result codes

$fetch_data->result();

Show all results of all calls (in the event of a redirect)

$fetch_data->result_raw();

Show the results of a specific call (in the event of a redirect)

$fetch_data->result_raw(0);

Table of Contents

Properties

$current_redirect  : int
$headers  : string
$keep_alive  : bool
$max_redirect  : int
$options  : array<string|int, mixed>
$post_data  : string
$response  : array<string|int, mixed>
$scheme_handlers  : array<string|int, mixed>
$user_options  : array<string|int, mixed>
$default_options  : array<string|int, mixed>
$still_alive  : array<string|int, mixed>

Methods

__construct()  : mixed
Start the curl object.
fetch()  : string|false
Get the contents of a URL, irrespective of allow_url_fopen.
request()  : object
Main calling function.
result()  : mixed
Used to return the results to the caller.
resultRaw()  : array<string|int, mixed>
Will return all results from all loops (redirects).
buildPostData()  : string
Takes supplied POST data and urlencodes it.
getRedirectUrl()  : string
Used if being redirected to ensure we have a fully qualified address.
headerCallback()  : int
Callback function to parse returned headers.
redirect()  : void
Called to initiate a redirect from a 301, 302 or 307 header.
sendRequest()  : void
Makes the actual curl call.
setOptions()  : void
Sets the final curl options for the current call.

Properties

$current_redirect

public int $current_redirect = 0

How many redirects have been followed.

$keep_alive

public bool $keep_alive = false

Whether to keep the connection open after the initial request.

$max_redirect

public int $max_redirect

Maximum number of redirects.

$options

public array<string|int, mixed> $options

An array of curl options.

$post_data

public string $post_data

Any post data as form name => value.

$response

public array<string|int, mixed> $response = []

Stores responses (url, code, error, headers, body, size).

$scheme_handlers

public static array<string|int, mixed> $scheme_handlers = ['ftp' => ['FtpFetcher'], 'ftps' => ['FtpFetcher'], 'http' => ['SocketFetcher', 'CurlFetcher'], 'https' => ['SocketFetcher', 'CurlFetcher']]

Specifies the fetcher class or classes to try for any given URL scheme.

Keys are URL schemes. Values are the names of one or more classes to try.

Class names will be prepended with NAMESPACE . '\APIs'.

$user_options

public array<string|int, mixed> $user_options = []

An array of curl options.

$default_options

private array<string|int, mixed> $default_options = [ // Get returned value as a string (don't output it). CURLOPT_RETURNTRANSFER => true, // We need the headers to do our own redirect. CURLOPT_HEADER => true, // Don't follow. We will do it ourselves so safe mode and open_basedir // will dig it. CURLOPT_FOLLOWLOCATION => false, // Set a normal looking user agent. CURLOPT_USERAGENT => SMF_USER_AGENT, // Don't wait forever on a connection. CURLOPT_CONNECTTIMEOUT => 15, // A page should load in this amount of time. CURLOPT_TIMEOUT => 90, // Accept gzip and decode it. CURLOPT_ENCODING => 'gzip,deflate', // Stop curl from verifying the peer's certificate. CURLOPT_SSL_VERIFYPEER => false, // Stop curl from verifying the peer's host. CURLOPT_SSL_VERIFYHOST => 0, // No post data. This will change if some is passed to request(). CURLOPT_POST => false, ]

The default curl options to use.

$still_alive

private static array<string|int, mixed> $still_alive = []

Fetchers that still have an open connection after the initial request.

Methods

__construct()

Start the curl object.

public __construct([array<string|int, mixed> $options = [] ][, int $max_redirect = 3 ]) : mixed
  • allow for user override values.
Parameters
$options : array<string|int, mixed> = []

An array of curl options.

$max_redirect : int = 3

Maximum number of redirects.

fetch()

Get the contents of a URL, irrespective of allow_url_fopen.

public static fetch(Url|string $url[, string|array<string|int, mixed> $post_data = [] ][, bool $keep_alive = false ]) : string|false
  • Reads the contents of an HTTP or FTP address and returns those contents in a string.
  • If $post_data is supplied, the value and length is posted to the given URL as form data.
Parameters
$url : Url|string

An HTTP or FTP URL.

$post_data : string|array<string|int, mixed> = []

The data to post to the given URL. Not applicable to FTP requests.

$keep_alive : bool = false

Whether to keep the connection alive for further requests. Not applicable to FTP requests.

Return values
string|false —

The fetched data or false on failure.

request()

Main calling function.

public request(string|Url $url[, array<string|int, mixed>|string $post_data = [] ]) : object
  • Will request the page data from a given $url.
  • Optionally will post data to the page form if $post_data is supplied. Passed arrays will be converted to a POST string joined with &'s.
  • Calls setOptions() to set the curl opts array values based on the defaults and user input.
Parameters
$url : string|Url

the site we are going to fetch

$post_data : array<string|int, mixed>|string = []

any post data as form name => value

Return values
object —

A reference to the object for method chaining.

result()

Used to return the results to the caller.

public result([string $area = null ]) : mixed
  • Called as ->result() will return the full final array.
  • Called as ->result('body') to return the page source of the result.
Parameters
$area : string = null

Used to return an area such as body, header, error.

Return values
mixed —

The response

resultRaw()

Will return all results from all loops (redirects).

public resultRaw([int $response_number = null ]) : array<string|int, mixed>
  • Can be called as ->result_raw(x) where x is a specific loop's result.
  • Call as ->result_raw() for everything.
Parameters
$response_number : int = null

Which response to get, or null for all.

Return values
array<string|int, mixed> —

The specified response or all the responses.

buildPostData()

Takes supplied POST data and urlencodes it.

protected buildPostData(array<string|int, mixed>|string $post_data) : string
  • Forms the data (for POST) into a string like var=xyz&var2=abc&var3=123
  • Drops vars with @ since we don't support sending files (uploading)
Parameters
$post_data : array<string|int, mixed>|string

The raw POST data.

Return values
string —

A string of POST data.

getRedirectUrl()

Used if being redirected to ensure we have a fully qualified address.

private getRedirectUrl([string $last_url = '' ][, string $new_url = '' ]) : string
Parameters
$last_url : string = ''

The URL we went to.

$new_url : string = ''

The URL we were redirected to.

Return values
string —

The new URL that was in the HTTP header.

headerCallback()

Callback function to parse returned headers.

private headerCallback(CurlHandle $cr, string $header) : int
  • Lowercases everything to make it consistent.
Parameters
$cr : CurlHandle

The curl request.

$header : string

The header.

Return values
int —

The length of the header.

redirect()

Called to initiate a redirect from a 301, 302 or 307 header.

private redirect(string $target_url, string $referrer_url) : void
  • Resets the curl options for the loop and sets the referrer flag.
Parameters
$target_url : string

The URL we want to redirect to.

$referrer_url : string

The URL that we're redirecting from.

sendRequest()

Makes the actual curl call.

private sendRequest(string $url[, bool $redirect = false ]) : void
  • Stores responses (url, code, error, headers, body) in the response array.
  • Detects 301, 302, 307 codes and will redirect to the given response header location.
Parameters
$url : string

The site to fetch.

$redirect : bool = false

Whether or not this was a redirect request.

setOptions()

Sets the final curl options for the current call.

private setOptions() : void
  • Overwrites our default values with user supplied ones or appends new user ones to what we have.
  • Sets the callback function now that $this is existing.

        

        
    




    

    

                                

                

                                
    

        On this page

        
    


                

                            

            

    

        

            
Search results