Storage Locations for files gathered by the Crawl Component sharepoint 2013

When gathering files from a content source, the SharePoint 2013 Crawl Component can be very I/O intensive process – locally writing all of the files it gathers from content repositories to its to temporary file paths and having them read by the Content Processing Component during document parsing. This post can help you understand where the Crawl Components write temporary files, which can help in planning and performance troubleshooting (e.g. Why does disk performance of my C:\ drive get so bad – or worse, fill up – when I start a large crawl?)

By default, all Search data files will be written within the Installation Path

  • The Data Directory (by default, a sub-directory of the Installation Path) specifies the path for all Search data files including those used by I/O intensive components (Crawl, Analytics, and Index Components)
    • The Data Directory can only be configured at the time of Installation (e.g. it can only be changed if uninstalling/re-installing SharePoint on the given server)
      • From the Installation Wizard, choose the “File Location” tab as seen below
      • IMPORTANT: Before uninstalling SharePoint, first modify your Search topology by removing any Search components from the applicable server. Once SharePoint is re-installed, you can once again deploy the components back to this server.
    • The defined path can be viewed in the registry:

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Setup\DataDirectory

    • Advanced Note: The Index files (by default, written to the Data Directory) path can be configured separately when provisioning an Index Component via PowerShell using the “RootDirectory” parameter

3175.installAndDataPath
(As a side note: the graphic is only intended to display the default locations specified at install time. It is recommended to change these to a file path other than C:\ drive)

For the Crawl Component:

  • When crawling [gathering] an item, the filter daemon (mssdmn.exe – a child process of the Crawl Component that actually interfaces with an end content repository using a Search Connector/Protocol Handler) will download any applicable file blobs to the SSA’s “TempPath” (e.g. an HTML file, a Word document, a PowerPoint presentation, etc)
    • In the graphic below, this is step 2a
    • The defined path can be viewed either:
      • In the registry (of a Crawl server)

        HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Global\Gathering Manager\TempPath

      • Or as a property of the SSA:

        $SSA = Get-SPEnterpriseSearchServiceApplication

        $SSA.TempPath

  • When the filter daemon completes the gathering of an item, it is returned to the Gathering Manager (mssearch.exe – responsible for orchestrating a crawl of a given item) and the applicable blob is moved to the “GathererDataPath“, which is a path relative to the DataDirectory mentioned above.
    • In the graphic below, this occurs in step 2b
    • The defined path can be viewed in the registry (of a Crawl server):

      HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Components\-GUID-of-theSSA-crawl-0\GathererDataPath

  • The GathererDataPath is mapped as a network share (used by the Content Processing Components)
    • The shared path can be viewed in the registry (of a Crawl server):

      HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\15.0\Search\Components\-GUID-of-theSSA-crawl-0\GathererDataShare

8233.crawlFlow
Usage by the Content Processing Components:

  • When the item is fed from the Crawler to the Content Processing Component (step 3 above), the item is only logically submitted to the CPC in a serialized payload of properties that represent that particular item – any related blob would remain on the Crawler and retrieved by a later stage in the processing flow
    • For SharePoint list items, there would typically not be a blob (unless the list item had an attachment)
    • For a document in a SharePoint library, the blob would represent the item’s associated file (such as a Word document)
  • During the Document Parsing stage in the processing flow (e.g. during step 4 above), the item’s blob will be retrieved from the Crawl Component via the GathererDataShare
  • When the Crawl Component receives a callback (success or failure) from the CPC (e.g. in step 6b above after an item has been processed), the temporary blob is then deleted from the GathererDataPath

1373.gathererDataShare
An example path to an item with DocID 933112 would look like the following:

file://crawlSrv/gthrsvc_7ecdbb10-3c86-4298-ab09-04f61aaeb636-crawl-0//f8/0xe3cf8_1.aspx   

#0xe3cf8 hex = 933112 decimal

Where:

  • crawlerSrv is a server running a crawl component
  • gthrsvc_-GUID-of-theSearchAdminWebServiceApp--crawl-0 is the name of the crawl component
    • This GUID can be identified using the following PowerShell:

      $SSA = Get-SPEnterpriseSearchServiceApplication

      $searchAdminWeb = Get-SPServiceApplication –Name $SSA.id

      $searchAdminWeb.id

      7ecdbb10-3c86-4298-ab09-04f61aaeb636

  • And the file name is actually re-named to the hex value of the docID
    • For example: 0xe3cf8 hex = 933112 decimal
    • Which we can see in ULS, such as:
      • From the Crawl Component (in this case, running on server “faceman”):

        mssearch.exe     SharePoint Server Search Crawler:Content Plugin      af7zf VerboseEx

        CTSDocument: FeedingDocument: properties : strDocID = ssic://933112 key = path values =\\FACEMAN\gthrsvc_7ecdbb10-3c86-4298-ab09-04f61aaeb636-crawl-0\\f8\0xe3cf8.aspx 

      • From the Content Processing Component:

        NodeRunnerContent2-834ebb1f-009    Search    Document Parsing      ai3ef VerboseEx

        AttachDocParser – Parsing: ‘file://faceman/gthrsvc_7ecdbb10-3c86-4298-ab09-04f61aaeb636-crawl-0//f8/0xe3cf8.aspx’

Advertisements

SharePoint Health Analyzer rules reference SharePoint 2013

Crawl error Processing this item failed because of an unknown error when trying to parse its contents sharepoint

During various search troubleshooting i came across the following crawling error in the Crawl log of a SharePoint 2013 environment.

Processing this item failed because of an unknown error when trying to parse its contents. (Error parsing document ‘http://********.*****.com/Project/abcd/Q_M/ABX/SitePages/Homepage.aspx’. Sandbox worker pool is
closed.; ; SearchID = *******************)

In order to fix this you can try to perform the following action plan:
Open “Local Policies
Click on “User rights assignment

user-rights-assignment

Make sure that the search service account has the following rights:
Replace a process level token

adjust-memory-quotas-for-process

Adjust memory quotas for a process

adjust-memory-quotas-for-process-properties

Impersonate a client after authentication

impersonate

Please make sure that the policies don’t get changed afterwards.

After implementing the above changes please run a clear configuration cache
After clearing the cache, start a full crawl and the errors should be gone.

Start SharePoint Service Application Proxy using Powershell

If your Usage and Health Data Collection Proxy is in a stopped state here is a quick bit of PowerShell to to get it started:

$sap = Get-SPServiceApplicationProxy | where-object {$_.TypeName -eq “Usage and Health Data Collection Proxy”}
$sap.Provision()

The above can easily be adapted to allow you to start any Service Application Proxy

Slow SharePoint improve performance without upgrading hardware

what you can do if your SharePoint is sometimes very slow.

E.g.: on the first start of a Site
Sometimes during the day a search query will take about a minute until you get results…..

Just look on that article: http://support.microsoft.com/kb/2625048

it will improve “feeled” performance (site response times) massive, if you’re going to implement both solutions.

Disabling CRL Check is just necessary if the SP Server does not have internet connectivity, that means proxy settings must be configured for the server itself

http://technet.microsoft.com/de-de/library/bb430772(v=exchg.141).aspx, and your proxy must allow traffic from the server of course.

SharePoint shortcut URL and hidden list

Users and Permissions:
People and Groups: _layouts/people.aspx
Site Collection Admins: _layouts/mngsiteadmin.aspx
Advanced Permissions: _layouts/user.aspx
Master Pages: _Layouts/ChangeSiteMasterPage.aspx
Look and Feel:
Quick Launch settings page: /_layouts/quiklnch.aspx
Title, Desc, and Icon: _layouts/prjsetng.aspx
Navigation: _layouts/AreaNavigationSettings.aspx
Page Layout and Ste Templates: _Layouts/AreaTemplateSettings.aspx
Welcome Page: _Layouts/AreaWelcomePage.aspx
Tree View: _layouts/navoptions.aspx
Top Nav Bar: _layouts/topnav.aspx
Site Theme: _layouts/themeweb.aspx
Reset to Site Definition: _layouts/reghost.aspx
Searchable Columns: _Layouts/NoCrawlSettings.aspx
Site Content Types: _layouts/mngctype.aspx
Galleries
Site Columns: _layouts/mngfield.aspx
Site Templates: _catalogs/wt/Forms/Common.aspx
List Templates: _catalogs/lt/Forms/AllItems.aspx
Filter toolbar for Lists and libraries: ?Filter=1
Web Parts: _catalogs/wp/Forms/AllItems.aspx
Workflows: _layouts/wrkmng.aspx
Workflow history hidden list: /lists/Workflow History
Master Pages and Page Layouts: _catalogs/masterpage/Forms/AllItems.aspx
Regoinal Settings: _layouts/regionalsetng.aspx
Site Administration
Recreate default site sp groups: _layouts/15/permsetup.aspx
recycle bin: _layouts/RecycleBin.aspx
Site Libraries and Lists: _layouts/mcontent.aspx
Site Usage Report: _layouts/usageDetails.aspx
User Alerts: _layouts/sitesubs.aspx
RSS: _layouts/siterss.aspx
Search Visibility: _layouts/srchvis.aspx
Sites and Workspaces: _layouts/mngsubwebs.aspx
Site Features: _layouts/ManageFeatures.aspx
Delete This Site: _layouts/deleteweb.aspx
Site Output Cache: _Layouts/areacachesettings.aspx
Content and Structure: _Layouts/sitemanager.aspx
Content and Structure Logs: _Layouts/SiteManager.aspx?lro=all
Search Settings: _layouts/enhancedSearch.aspx
Site Collection Administration
Search Scopes: _layouts/viewscopes.aspx?mode=site
Search Keywords: _layouts/listkeywords.aspx
Recycle Bin: _layouts/AdminRecycleBin.aspx
Site Collection Features: _layouts/ManageFeatures.aspx?Scope=Site
Site Hierachy: _layouts/vsubwebs.aspx
Site hierarchy page (lists of sub sites): /_layouts/1033/vsubwebs.aspx
Portal Site Connection: _layouts/portal.aspx
Site Collection Audit Settings: _layouts/AuditSettings.aspx
Site Collection Policies: _layouts/Policylist.aspx
Site Collection Cache Profiles: Cache%20Profiles/AllItems.aspx
Site Collection Output Cache: _Layouts/sitecachesettings.aspx
Site Collection Object Cache: _Layouts/objectcachesettings.aspx
Variations: _Layouts/VariationSettings.aspx
Variation Labels: _Layouts/VariationLabels.aspx
Translatable Columns: _Layouts/TranslatableSettings.aspx
Variation Logs: _Layouts/VariationLogs.aspx
Site Settings: _layouts/settings.aspx
Delete user from Site collection (on-premises): /_layouts/15/people.aspx?MembershipGroupId=0

Load document tab initial
?InitialTabId=Ribbon.Document

Delete user from Site collection (on-premises):
/_layouts/15/people.aspx?MembershipGroupId=0

Display list in grid view. ‘True’ is case sensitive:
?ShowInGrid=True

Sandboxed Solution Gallery:
/_catalogs/solutions/Forms/AllItems.aspx

Filter toolbar for Lists and libraries:
?Filter=1

Site usage page:
/_layouts/usage.aspx

View all site content page (Site content):
/_layouts/viewlsts.aspx

Get the version of the SharePoint server (Patch level):
/_vti_pvt/Service.cnf

Web Part Maintenance Page:
?Contents=1

Show Page in Dialog View:
?isdlg=1

Application page for registering SharePoint apps
/_layouts/15/appregnew.aspx

Save Site as a template
/_layouts/savetmpl.aspx

Sign in as a different user
/_layouts/closeConnection.aspx?loginasanotheruser=true

Enable SharePoint designer
/_layouts/SharePointDesignerSettings.aspx

Quick Deploy List
Quick%20Deploy%20Items/AllItems.aspx

Open Page in Edit Mode
?ToolPaneView=2

Taxonomy Hidden List (MMS)
Lists/TaxonomyHiddenList/AllItems.aspx

User Information List:
_catalogs/users
_catalogs/users/simple.aspx

Force displaying the user profile in the site collection:
/_layouts/userdisp.aspx?id={UserID}&Force=True

Site hierarchy page (lists of sub sites)
/_layouts/vsubwebs.aspx
/_layouts/1033/vsubwebs.aspx

Add Web Parts Pane: ?ToolPaneView=2 : Add to the end of the page URL; will only work if the page is already checked out
Create: [area]/_layouts/spscreate.aspx
Create: /_layouts/create.aspx

Create list in a different portal area :

/_layouts/new.aspx?NewPageFilename=YourTemplateName.stp&ListTemplate=100&
ListBaseType=0

When you save a template in a portal area and try to create a new list in a different portal area, the template will not show on the Create page. Use this URL to force it to show.

Documents and Lists: /_layouts/viewlsts.aspx

List Template Gallery: /_catalogs/lt

Manage Audiences: /_layouts/Audience_Main.aspx

Manage Cross Site Groups: /_layouts/mygrps.aspx

Manage List Template Gallery: /_catalogs/lt/Forms/AllItems.aspx

Manage My Alerts: /_layouts/MySubs.aspx

Manage People: /_layouts/people.aspx

Manage Site Collection Administrators: /_layouts/mngsiteadmin.aspx

Manage Site Collection Users:
/_layouts/siteusrs.aspx : To access you must be an admin on the server or a site collection admin for the site.

Manage Site Groups: /_layouts/role.aspx

Manage Site Template Gallery: /_catalogs/wt/Forms/AllItems.aspx

Manage Site Template Gallery: /_catalogs/wt/Forms/Common.aspx

Manage Sites and Workspaces: /_layouts/mngsubwebs.aspx

Manage User Alerts: /_layouts/AlertsAdmin.aspx

Manage User Alerts: /_layouts/SiteSubs.aspx

Manage User Permissions: /_layouts/user.aspx

Manage Web Part Gallery: /_catalogs/wp/Forms/AllItems.aspx

Master Page Gallery: /_catalogs/masterpage : Also includes Page Layouts

Modify Navigation: /_layouts/AreaNavigationSettings.aspx

Recycle Bin: /_layouts/AdminRecycleBin.aspx

Save as site template: /_layouts/savetmpl.aspx

Site Column Gallery: /_layouts/mngfield.aspx

Site Content and Structure Manager: /_layouts/sitemanager.aspx

Site Content Types: /_layouts/mngctype.aspx

Site Settings: /_layouts/settings.aspx

Site Settings: /_layouts/default.aspx

Site Template Gallery: /_catalogs/wt

Site Theme: /_layouts/themeweb.aspx

Site usage details: /_layouts/UsageDetails.aspx

Site Usage Summary: /_layouts/SpUsageWeb.aspx

Site Usage Summary: /_layouts/Usage.aspx

Sites Registry: /SiteDirectory/Lists/Sites/Summary.aspx

Top-level Site Administration: /_layouts/webadmin.aspx

User Information: /_layouts/userinfo.aspx

Web Part Gallery: /_catalogs/wp

Web Part Page Maintenance: ?contents=1 : Add to the end of the page URL

Display template SharePoint Server 2013

Display templates for the Content Search Web Part

You can use the following display templates to change the appearance of content that is shown in a Content Search Web Part. These display template files are located in the Content Web Parts subfolder in the Display Templates folder in the Master Page Gallery.

template-contentsearchwebpart

Display templates for the Refinement Web Part and the Taxonomy Refinement Web Part

You can use the display templates listed in the following table to change the appearance of content that is shown in a Refinement Web Part and a Taxonomy Refinement Web Part. These display template files are located in the Filters subfolder in the Display Templates folder in the Master Page Gallery. Note that there are different display templates for different refiner types.

template-webpart

Display templates for the Search Results Web Part

You can use the display templates in the following table to change the appearance of content shown in a Search Results Web Part. Note that the hover panels for the different result types have separate display templates. These display template files are located in the Search subfolder in the Display Templates folder in the Master Page Gallery.

template-search1
template-search2

HTTP 403 Forbidden error when try browse to a SharePoint web app

Received the following error when browse to a SharePoint web app

The website declined to show this webpage
HTTP 403
Most likely causes:
This website requires you to log in.

http-403

if we create a copy of the web.config file, rename the web.config file, refresh the home page, we receive an “HTTP 404 – Page Not Found” error.

Rename the web.config file back and refresh the page. The site is browse able for a while before failing after some time, We see the following error in Failed Request Tracing

filed-request-tracing

A procmon trace captured while accessing the web app from the server showed the following:

w3wp.exe 4180 CreateFile

C:\inetpub\wwwroot\wss\VirtualDirectories\Web80.Contoso.com80\bin ACCESS DENIED Desired Access: Read Data/List Directory, Synchronize
Disposition: Open
Options: Directory, Synchronous IO Non-Alert
Attributes: n/a
ShareMode: Read, Write, Delete
AllocationSize: n/a
Impersonating: NT AUTHORITY\IUSR

tcs-view

This issue usually occurs when a request from an authenticated user without local admin rights results in a failed read of the /BIN directory by the impersonating w3wp.exe (IIS worker process for ASP.NET) process.

This behavior is typically associated with lack of permissions to the temporary folder /BIN where ASP.Net assemblies are Just In Time (JIT) compiled.

Resolution

The solution is to ensure that the Authenticated Users or \Users group (which usually contains DOMAIN\Users group) has Read & Execute, List Folder Contents and Read permissions on the /BIN folder below

C:\inetpub\wwwroot\wss\VirtualDirectories{Sitename80}.

Follow the steps below to grant the required permissions:

a. Open Windows Explorer and navigate to the /bin directory of your web application
b. Right-click on the folder and click on Properties
c. Go to Security tab and click on Edit
d. Click on Add and add the local server group Authenticated Users or \Users (this usually contains DOMAIN\Users group).
e. Select the Read & Execute, List Folder Contents and Read permissions (if you are planning to add Everyone to the /bin folder, grant Read permissions only)
f. Click OK to apply the new settings
g. Refresh the page and we should be able to browse to the site.

More Information

If an administrator accesses the site/feature that caused the error, the subsequent requests from non-administrators would succeed. This behavior is typically associated with lack of permissions to the temporary folder where ASP.Net assemblies are Just In Time compiled.

The freb trace shows a 403.0 for ManagedPipelineHandler

It seems to go through quite a few ASPNet events – but happens during the ASPNetPageRender – it goes to the ASPNetPageRender Enter, then ASPNetHTTPHandler Leave.Only then does it get a 403.0 which is not an official RFC error. The first sub-status for 403 is 403.0.

Application pool in Classic or Integrated mode

Application Pool in Classic Mode – In this case, we can configure a Wildcard mapping for ASPNET_ISAPI.dll at the website level. That would propagate to child virtual directories. That should not need any further modifications at the virtual directory level.

Application Pool in Integrated Mode – In this case, all relevant virtual directories would need individual modifications. They need to be set for specific handler.

 

Networking Utilities tools helpful for sharepoint

Active Directory Explorer:

Active Directory Explorer (AD Explorer) is an advanced Active Directory (AD) viewer and editor. We can use AD Explorer to easily navigate an AD database, define favorite locations, view object properties and attributes without having to open dialog boxes, edit permissions, view an object’s schema, and execute sophisticated searches that you can save and re-execute.

AD Explorer also includes the ability to save snapshots of an AD database for off-line viewing and comparisons. When we load a saved snapshot, we can navigate and explore it as we would a live database. If we have two snapshots of an AD database we can use AD Explorer’s comparison functionality to see what objects, attributes and security permissions changed between them.

active-directory-explorer
download

Insight for Active Directory:

ADInsight is an LDAP (Light-weight Directory Access Protocol) real-time monitoring tool aimed at troubleshooting Active Directory client applications. Use its detailed tracing of Active Directory client-server communications to solve Windows authentication, Exchange, DNS, and other problems.

ADInsight uses DLL injection techniques to intercept calls that applications make in the Wldap32.dll library, which is the standard library underlying Active Directory APIs such ldap and ADSI.

Unlike network monitoring tools, ADInsight intercepts and interprets all client-side APIs, including those that do not result in transmission to a server.

ADInsight monitors any process into which it can load it’s tracing DLL, which means that it does not require administrative permissions, however, if run with administrative rights, it will also monitor system processes, including windows services.

insight-for-active-directory

download
AdRestore v1.1:

Windows Server 2003 introduces the ability to restore deleted (“tombstoned”) objects. This simple command-line utility enumerates the deleted objects in a domain and gives you the option of restoring each one. Source code is based on sample code in the Microsoft Platform SDK. This MS KB article describes the use of AdRestore:

840001: How to restore deleted user accounts and their group memberships in Active Directory

adrestore

download

PipeList v1.02:

Did you know that the device driver that implements named pipes is actually a file system driver” In fact, the driver’s name is NPFS.SYS, for “Named Pipe File System”. What you might also find surprising is that its possible to obtain a directory listing of the named pipes defined on a system. This fact is not documented, nor is it possible to do this using the Win32 API. Directly using NtQueryDirectoryFile, the native function that the Win32 FindFile APIs rely on, makes it possible to list the pipes. The directory listing NPFS returns also indicates the maximum number of pipe instances set for each pipe and the number of active instances.

downloadPsFile v1.02:

Introduction

The “net file” command shows you a list of the files that other computers have opened on the system upon which you execute the command, however it truncates long path names and doesn’t let you see that information for remote systems.

PsFile is a command-line utility that shows a list of files on a system that are opened remotely, and it also allows you to close opened files either by name or by a file identifier.

Installation

Just copy PsFile onto your executable path, and type “psfile”.

Using PsFile

The default behavior of PsFile is to list the files on the local system that are open by remote systems. Typing a command followed by “- ” displays information on the syntax for the

command.

Usage: psfile [\RemoteComputer [-u Username [-p Password]]] [[Id | path] [-c]]

-u Specifies optional user name for login to remote computer.
-p Specifies password for user name. If this is omitted, you will be prompted to enter the password without it being echoed to the screen.
Id Identifier (as assigned by PsFile) of the file for which to display information or to close.
Path Full or partial path of files to match for information display or close.
-c Closes the files identifed by ID or path.

How it Works

PsFile uses the NET API, which is documented in the Platform SDK.download

PsPing v2.01:

Introduction

PsPing implements Ping functionality, TCP ping, latency and bandwidth measurement. Use the following command-line options to show the usage for each test type:

Installation
Copy PsPing onto your executable path. Typing “psping” displays its usage syntax.

Using PsPing
PsPing implements Ping functionality, TCP ping, latency and bandwidth measurement. Use the following command-line options to show the usage for each test type:

Usage: psping -? [i|t|l|b]

-? I Usage for ICMP ping.
-? T Usage for TCP ping.
-? L Usage for latency test.
-? B Usage for bandwidth test.

ICMP ping usage: psping [[-6]|[-4]] [-h [buckets | ,,…]] [-i ] [-l [k|m] [-q] [-t|-n ] [-w ]

-h Print histogram (default bucket count is 20).
If you specify a single argument, it’s interpreted as a bucket count and the histogram will contain that number of buckets covering the entire time range of values. Specify a comma-separated list of times to create a custom histogram (e.g. “0.01,0.05,1,5,10”).
-i Interval in seconds. Specify 0 for fast ping.
-l Request size. Append ‘k’ for kilobytes and ‘m’ for megabytes.
-n Number of pings or append ‘s’ to specify seconds e.g. ’10s’.
-q Don’t output during pings.
-t Ping until stopped with Ctrl+C and type Ctrl+Break for statistics.
-w Warmup with the specified number of iterations (default is 1).
-4 Force using IPv4.
-6 Force using IPv6.
For high-speed ping tests use -q and -i 0.

TCP ping usage: psping [[-6]|[-4]] [-h [buckets | ,,…]] [-i ] [-l [k|m] [-q] [-t|-n ] [-w ]

-h Print histogram (default bucket count is 20).
If you specify a single argument, it’s interpreted as a bucket count and the histogram will contain that number of buckets covering the entire time range of values. Specify a comma-separated list of times to create a custom histogram (e.g. “0.01,0.05,1,5,10”).
-i Interval in seconds. Specify 0 for fast ping.
-l Request size. Append ‘k’ for kilobytes and ‘m’ for megabytes.
-n Number of pings or append ‘s’ to specify seconds e.g. ’10s’.
-q Don’t output during pings.
-t Ping until stopped with Ctrl+C and type Ctrl+Break for statistics.
-w Warmup with the specified number of iterations (default is 1).
-4 Force using IPv4.
-6 Force using IPv6.
For high-speed ping tests use -q and -i 0.

TCP and UDP latency usage:

server: psping [[-6]|[-4]] [-f]
client: psping [[-6]|[-4]] [-f] [-u] [-h [buckets | ,,…]] [-r] [k|m]] [-w ]

-f Open source firewall port during the run.
-u UDP (default is TCP).
-h Print histogram (default bucket count is 20).
If you specify a single argument, it’s interpreted as a bucket count and the histogram will contain that number of buckets covering the entire time range of values. Specify a comma-separated list of times to create a custom histogram (e.g. “0.01,0.05,1,5,10”).
-l Request size. Append ‘k’ for kilobytes and ‘m’ for megabytes.
-n Number of sends/receives. Append ‘s’ to specify seconds e.g. ’10s’
-r Receive from the server instead of sending.
-w Warmup with the specified number of iterations (default is 5).
-4 Force using IPv4.
-6 Force using IPv6.
-s Server listening address and port.

The server can serve both latency and bandwidth tests and remains active until you terminate it with Control-C.

TCP and UDP bandwidth usage:

server: psping [[-6]|[-4]] [-f]
client: psping [[-6]|[-4]] [-f] [-u] [-h [buckets | ,,…]] [-r] [k|m]] [-i ] [-w ]

-f Open source firewall port during the run.
-u UDP (default is TCP).
-b Bandwidth test.
-h Print histogram (default bucket count is 20).
If you specify a single argument, it’s interpreted as a bucket count and the histogram will contain that number of buckets covering the entire time range of values. Specify a comma-separated list of times to create a custom histogram (e.g. “0.01,0.05,1,5,10”).
-i Number of outstanding I/Os (default is min of 16 and 2x CPU cores).
-l Request size. Append ‘k’ for kilobytes and ‘m’ for megabytes.
-n Number of sends/receives. Append ‘s’ to specify seconds e.g. ’10s’
-r Receive from the server instead of sending.
-w Warmup for the specified iterations (default is 2x CPU cores).
-4 Force using IPv4.
-6 Force using IPv6.
-s Server listening address and port.

The server can serve both latency and bandwidth tests and remains active until you terminate it with Control-C.

Examples
This command executes an ICMP ping test for 10 iterations with 3 warmup iterations:

psping -n 10 -w 3 marklap

To execute a TCP connect test, specify the port number. The following command executes connect attempts against the target as quickly as possible, only printing a summary when finished with the 100 iterations and 1 warmup iteration:

psping -n 100 -i 0 -q marklap:80

To configure a server for latency and bandwidth tests, simply specify the -s option and the source address and port the server will bind to:

psping -s 192.168.2.2:5000

A buffer size is required to perform a TCP latency test. This example measures the round trip latency of sending an 8KB packet to the target server, printing a histogram with 100 buckets when completed:

psping -l 8k -n 10000 -h 100 192.168.2.2:5000

This command tests bandwidth to a PsPing server listening at the target IP address for 10 seconds and produces a histogram with 100 buckets. Note that the test must run for at least one second after warmup for a histogram to generate. Simply add -u to have PsPing perform a UDP bandwidth test.

psping -b -l 8k -n 10000 -h 100 192.168.2.2:5000

download

PsTools:

Introduction

The Windows NT and Windows 2000 Resource Kits come with a number of command-line tools that help you administer your Windows NT/2K systems. Over time.

The first tool in the suite was PsList, a tool that lets you view detailed information about processes, and the suite is continually growing.

The “Ps” prefix in PsList relates to the fact that the standard UNIX process listing command-line tool is named “ps”, so adopted this prefix for all the tools in order to tie them together into a suite of tools named PsTools.

Note: some anti-virus scanners report that one or more of the tools are infected with a “remote admin” virus. None of the PsTools contain viruses, but they have been used by viruses, which is why they trigger virus notifications.

The tools included in the PsTools suite, which are downloadable as a package, are:

PsExec – execute processes remotely
PsFile – shows files opened remotely
PsGetSid – display the SID of a computer or a user
PsInfo – list information about a system
PsPing – measure network performance
PsKill – kill processes by name or process ID
PsList – list detailed information about processes
PsLoggedOn – see who’s logged on locally and via resource sharing (full source is included)
PsLogList – dump event log records
PsPasswd – changes account passwords
PsService – view and control services
PsShutdown – shuts down and optionally reboots a computer
PsSuspend – suspends processes
PsUptime – shows you how long a system has been running since its last reboot (PsUptime’s functionality has been incorporated into PsInfo)
The PsTools download package includes an HTML help file with complete usage information for all the tools.

download

ShareEnum v1.6

Introduction

An aspect of Windows NT/2000/XP network security that’s often overlooked is file shares. A common security flaw occurs when users define file shares with lax security, allowing unauthorized users to see sensitive files.

There are no built-in tools to list shares viewable on a network and their security settings, but ShareEnum fills the void and allows you to lock down file shares in your network.

When you run ShareEnum it uses NetBIOS enumeration to scan all the computers within the domains accessible to it, showing file and print shares and their security settings. Because only a domain administrator has the ability to view all network resources, ShareEnum is most effective when you run it from a domain administrator account.

shareenum

How It Works

ShareEnum uses WNetEnumResource to enumerate domains and the computers within. them and NetShareEnum to enumerate shares on computers.

download

TCPView v3.05

Introduction

TCPView is a Windows program that will show you detailed listings of all TCP and UDP endpoints on your system, including the local and remote addresses and state of TCP connections. On Windows Server 2008, Vista, and XP, TCPView also reports the name of the process that owns the endpoint.

TCPView provides a more informative and conveniently presented subset of the Netstat program that ships with Windows. The TCPView download includes Tcpvcon, a command-line version with the same functionality.

tcpview

Using TCPView

When you start TCPView it will enumerate all active TCP and UDP endpoints, resolving all IP addresses to their domain name versions. You can use a toolbar button or menu item to toggle the display of resolved names. On Windows XP systems, TCPView shows the name of the process that owns each endpoint.

By default, TCPView updates every second, but you can use the Options|Refresh Rate menu item to change the rate. Endpoints that change state from one update to the next are highlighted in yellow; those that are deleted are shown in red, and new endpoints are shown in green.

You can close established TCP/IP connections (those labeled with a state of ESTABLISHED) by selecting File|Close Connections, or by right-clicking on a connection and choosing Close Connections from the resulting context menu.
You can save TCPView’s output window to a file using the Save menu item.

Using Tcpvcon

Tcpvcon usage is similar to that of the built-in Windows netstat utility:
Usage: tcpvcon [-a] [-c] [-n] [process name or PID]

-a Show all endpoints (default is to show established TCP connections).
-c Print output as CSV.
-n Don’t resolve addresses..

Microsoft TCPView KB Article

This Microsoft KB article references TCPView:
816944: “Unexpected Error 0x8ffe2740 Occurred” Error Message When You Try to Start a Web Site

download

Whois v1.14

Introduction

Whois performs the registration record for the domain name or IP address that you specify.

Usage

Usage: whois [-v] domainname [whois.server]

-v Print whois information for referrals
Domainname can be either a DNS name (e.g. http://www.sysinternals.com) or IP address (e.g. 66.193.254.46).

download